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PREFACE 


The  Department  of  Defense  (DoD)  has  a  long  history  of  problems  with  the  acquisition  and 
support  of  test  equipment.  In  a  previous  report,  Test  Equipment  Management.  January  1985,  we 
summarized  the  nature  and  extent  of  the  underlying  problems  and  recommended  the  Assistant 
Secretary  of  Defense  (Manpower,  Installations,  and  Logistics),  ASD1MI&L),  take  the  lead  in  effecting 
needed  improvements.  The  ASD(MI&L)  concurred  with  that  recommendation  and  established  a 
"DoD  Test  Equipment  Management  Improvement  Program”  under  the  overall  guidance  of  the 
Maintenance  Directorate.  This  report  recommends  a  set  of  actions  for  that  program. 

The  report  is  published  in  four  volumes.  Volume  I  presents  a  program  of  action  for  improving 
test  equipment  management  and  support  within  the  DoD.  Volume  II  reviews  previous  studies  and 
initiatives  pertaining  to  test  equipment  management  and  support.  Volume  III  describes  how  the 
Military  Services  are  organized  to  carry  out  that  management  and  support.  Volume  IV  reviews  and 
assesses  the  adequacy  of  related  DoD  policy. 

Throughout  the  report,  all  references  to  military  organizations  apply  to  the  situation  in  early 
1985.  Subsequent  organizational  changes,  such  as  the  Navy’s  disestablishment  of  the  Naval  Material 
Command  and  the  reorganization  of  Naval  Electronic  Systems  Command  into  Space  and  Naval 
Warfare  Systems  Command,  are  not  reflected  in  the  text.  As  a  result,  several  old  office  symbols  and 
references  are  used.  Similarly,  several  events  that  have  taken  place  after  mid  1985  with  regard  to 
test  equipment  management  and  support  may  not  be  included 
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1.  INTRODUCTION 


This  volume  reviews  and  comments  on  the  major  studies  undertaken  in  recent  years  to  improve 
test  equipment  management  and  support.  Its  purpose  is  to  illustrate  the  complexity  and  persistence 
of  the  'test  equipment  problem’4;  the  different  solutions  emphasized  by  the  Military  Services;  the 
difficulties  encountered  in  implementing  joint  or  coordinated  actions;  and,  by  implication,  the 
problem  areas  that  either  have  not  been  addressed  or  are  receiving  inadequate  emphasis  within  the 
Department  of  Defense  (DoD). 

X  . 

TERMINOLOGY  V 

Each  of  the  Military  Services,  as  well  as  the  Office  of  the  Secretary  of  Defense  (OSD),  uses  its 
own  nomenclature  for  test  equipment.  Throughout  this  report,  we  use  the  term  "test  equipment”  as  a 
specific  category  of  support  equipment;  i.e.,  electric/electronic  test,  measurement,  and  diagnostic 
equipment  (TMDE),  defined  to  include  manual  electronic  test  equipment  (ETE),  automatic  test 
equipment  (ATE),  test  program  sets  (TPSs),  and  calibration  equipment  (reference  and  transfer 
standards  and  associated  equipment).  Each  of  these  four  categories  of  test  equipment  can  be  further 
subdivided  as  shown  in  our  taxonomy  for  test  equipment  classification  (Figure  11)  That  taxonomy  is 
somewhat  nonstandard  in  that  the  term  TMDE,  as  used  by  the  Military  Services,  frequently  includes 
built-in  test  software/equipment  (BIT/BITE)  —  a  category  beyond  the  focus  of  this  report.  We  also 
exclude  two  other  categories  of  test  equipment:  automated  job  performance  aids  (support  equipment 
that  differs  from  test  equipment  in  that  it  is  not  hooked  up  to  the  prime  equipment)  and  physical/ 
mechanical  test  equipment.  While  the  latter  represents  a  significant  portion  of  the  test  equipment 
inventory,  its  management  and  support  are  considerably  simpler  than  those  for  electric/electronic 
TMDE 

ORGANIZATION 

This  review  of  key  studies  and  initiatives  is  organized  by  sponsoring  organizations,  starting 
with  OSD  (Chapter  2),  the  Congress  (Chapter  3),  the  Joint  Logistics  Commanders,  JLC,  (Chapter  4). 


SUPPLEMENTARY  DATA 


2.  OSD  EFFORTS 


The  OSD  has  sponsored  three  major  efforts  aimed  at  improving  test  equipment  management: 
(l)a  study  conducted  from  1974  to  1976  by  the  Defense  Science  Board  Task  Force  on  Electronic  Test 
Equipment  (also  known  as  the  "Fluke  Committee”  after  the  chairman  of  the  task  force)  and 
associated  efforts  to  implement  the  recommendations.  (2)  the  activities  of  the  "DoD  Ad  Hoc  TMDE 
Standardization  Working  Group,”  which  was  chartered  in  1980;  and  (3)  the  actions  undertaken  to 
implement  Acquisition  Improvement  Program  Initiative  #30,  "Improved  Management  of  Support 
Resources  for  Selected  Weapon  Systems,”  beginning  in  1981 
DEFENSE  SCIENCE  BOARD  STUDY 

In  October  of  1974,  the  Director  of  Defense  Research  and  Engineering  and  the  Assistant 
Secretary  of  Defense  (Installations  and  Logistics),  requested  the  Defense  Science  Board  to  establish  a 
task  force  to  review  acquisition  and  management  of  ETE  within  the  DoD.  The  task  force  was 
chartered  to  examine  the  potential  for  greater  use  by  the  DoD  of  commercially  available  ETE  in  order 
to  achieve  economy  and  reliability  benefits  over  ETE  built  to  military  specifications,  and  to  recom¬ 
mend  policies  and  procedures  that  would  maximize  these  benefits.  After  conducting  its  review,  the 
task  force,  chaired  by  John  M  Fluke,  Sr  ,  and  composed  of  representatives  of  industry.  Military 
Services,  and  the  OSD,  submitted  a  report  to  the  Secretary  of  Defense  1  That  report  contained 
28  major  recommendations  addressing  problems  in  four  broad  areas  requirements,  procurement, 
logistics,  and  management 

The  task  force  defined  ETE  to  be  "all  electronic  devices  used  to  measure,  gauge,  test,  inspect, 
diagnose  or  otherwise  examine  materials,  supplies,  and  equipment  to  determine  compliance  with 

■The  Task  Force  on  Electronic  Test  Equipment,  "Use  of  Off  the  Shelf  Electronic  Test  Equip 
ment  to  Reduce  Costs,  Shorten  Leadtimes,  Assure  Reliability  and  Simplify  Logistics"  (Washington. 
1)  C  Defense  Science  Board,  Office  of  the  Director  of  Defense  Research  and  Engineering.  Fehruarv 
1976) 


requirements  established  in  technical  documents."  It  further  subdivided  ETE  into  off-the-shelf  ETE, 
which  is  equipment  in  regular  commercial  production,  and  modified  off-the-shelf  ETE,  which  is 
equipment  in  regular  production  but  modified,  however  slightly,  to  meet  DoD  specifications. 


The  recommendations  of  the  task  force,  by  problem  area,  are  presented  below. 

Requirements 

The  task  force  recommended  several  actions  to  improve  the  ETE  requirements  process. 
First,  it  recommended  that  early  major  system  reviews  consider  the  support  costs  of  the  associated 
ETE  needed  to  perform  mission  requirements.  It  also  recommended  that  DoD  policies  and  procedures 
for  acquisition  and  support  of  ETE,  which  encouraged  use  of  the  system  prime  contractor  and  sub¬ 
contractors,  be  reexamined  to  assure  that  this  approach  was  the  most  economic 

The  task  force  found  that  the  Military  Services  were  not  sharing  the  "lessons  learned”  in 
major  weapons  system  ETE.  To  correct  this  situation,  it  recommended  that  formal  procedures  be 
established  to  promote  sharing  of  "lessons  learned,"  both  within  and  among  Military  Departments. 

The  task  force  also  found  that  the  Military  Services  frequently  used  military  specifi¬ 
cations  for  ETE  when  off-the-shelf  equipment  with  similar  capabilities  was  available.  As  a  conse¬ 
quence,  they  incurred  increased  acquisition  and  support  costs,  delayed  delivery,  increased  likelihood 
of  developing  obsolete  ETE,  and  more  operational,  maintenance,  and  calibration  problems.  As 
corrective  action,  the  task  force  recommended  that  the  use  of  military  specifications  in  areas  where 
off-the-shc  .f  ETE  can  meet  the  requirements  be  justified  and  reviewed  It  further  recommended  that 
military  specification  MIL-T-28800C  ("Test  Equipment  for  Use  with  Electrical  and  Electronic  Equip¬ 
ment,  General  Specification  for”)  be  revised  to  encourage  procurement  of  off-the-shelf  ETE. 

To  improve  the  compatibility,  versatility,  and  usability  of  ATE,  the  task  force  recom¬ 
mended  that  joint  DoD/industrv  guidelines  be  developed  for  the  design,  development,  acquisition, 
use,  and  support  of  ATE  The  task  force  also  recommended  that  the  DoD  include  testability  as  a 
design  consideration  in  weapons  systems  and  subsystems,  select  a  common  ATE  software  test  lan¬ 
guage,  and  require  that  ATE  purchases  be  justified  on  both  economic  and  technical  grounds 


ETE  manufacturers  routinely  obtain  commercial  market  information  by  soliciting 
potential  customers,  attending  electronics  industry  meetings,  and  studying  advances  in  electronics 
technology.  Those  manufacturers  cannot  use  the  same  techniques  to  anticipate  the  DoD’s  require¬ 
ments.  In  recognition  of  this  deficiency,  the  task  force  recommended  that  a  formal  program  be  under¬ 
taken  to  define  equipment  needs  early  and  make  the  information  available  to  industry. 

Procurement 

In  the  area  of  procurement,  the  task  force  found  that  the  procedures  for  DoD  purchase  of 
off-the-shelf  ETE  were  unnecessarily  complex  and  costly.  It  recommended  use  of  multiple-award 
Federal  Supply  Schedule  purchase  methods  to  reduce  costs,  save  time,  and  simplify  the  process.  A 
minority  recommended  that  the  multiple-award  Federal  Supply  Schedule  method  be  required  for 
purchase  of  off-the-shelf  ETE. 

The  task  force  also  found  that  contracts  for  off-the-shelf  ETE  were  generally  awarded  to 
the  lowest  bidder  in  acquisition  cost  rather  than  to  the  lowest  life-cycle  cost  offerer.  To  correct  this 
situation,  it  recommended  that  the  guidelines  for  evaluating  ownership  costs  of  ETE  be  improved  and 
that  those  costs,  in  addition  to  bid  price,  be  considered  in  awarding  contracts 

To  provide  increased  flexibility  in  the  procurement  of  ETE,  the  task  force  recommended 
that  DoD  procurement  policies  and  practices  be  revised  to  encourage  the  use  of  bid  samples  and  to 
allow  consideration  of  a  manufacturer's  capability  to  provide  spare  parts  as  well  as  technical  and 
service  support  after  the  ETE  is  purchased 

To  reduce  the  proliferation  of  different  makes  and  models  of  ETE,  the  task  force  recom¬ 
mended  that  the  DoD  prepare  guidelines  for  standardizing  on  specific  off-the-shelf  ETE  by  taking  into 
account  operating  and  support  costs,  obsolescence,  and  technology  improvements  in  ETE  The  task 
force  noted  that  preferred  items  lists  (PlLsl  is  one  method  of  reducing  proliferation,  but  the  use  of 
such  lists  should  not  limit  purchases  to  one  manufacturer's  equipment  when  other  off- the  shelf  ETE 
is  available  to  perform  the  same  function  A  minority  r  commended  that  PILs  refer  primarily  to 
specification  requirements  and  not  to  a  single  manufacturer's  piece  of  equipment  unless  it  is  military 
specification  equipment  with  Government  owned  rights  and  manufacturing  data 


The  task  force  also  found  that  DoD  supply  systems  and  administrative  procedures  make 
it  difficult  for  the  DoD  to  take  advantage  of  manufacturers’  warranties  It  recommended  that  those 
procedures  be  revised  to  assure  that  the  DoD  obtains  maximum  benefit  from  warranties  It  further 
recommended  that  DoD  policy  be  revised  to  specifically  call  for  enforcement  of  contract  terms  and  use 
of  remedies  to  establish  the  expectation  of  compliance  by  manufacturers  In  addition,  to  minimize  the 
processing  of  claims  by  manufacturers  of  off-the-shelf  ETE,  the  DoD  should  modify  its  contracting 
regulations  to  simplify  the  assignment  of  receivables  on  contracts  up  to  $250,000  at  the  contractor’s 
discretion  under  the  Uniform  Commercial  Code. 

Logistics 

In  the  area  of  logistics,  the  task  force  found  that  the  Military  Services  fail,  in  many 
cases,  to  take  advantage  of  commercial  support  resources,  including  repair  parts,  repair  and 
calibration  services,  and  training.  To  correct  this  situation,  the  task  force  recommended  the  DoD 
issue  revised  policy  which  requires  the  Military  Services  to  increase  their  use  of  commercial  resources 
to  support  ETE  when  economical  and  effective.  They  also  should  be  authorized  and  funded  to 
establish  a  pool  of  selected  ETE  end  items  at  appropriate  locations  to  replace  equipment  temporarily 
out  of  service 

The  task  force  also  found  that  the  DoD  needs  to  place  a  higher  priority  on  surveying  the 
resources,  costs,  and  utilization  of  military  ETE  calibration  and  repair  facilities  The  task  force 
recommended  that  those  facilities  be  consolidated  or  disestablished,  whenever  possible,  and  that 
procedures  be  developed  to  encourage  the  use  of  ETE  manufacturers  in  calibration  and  repair, 
possibly  through  the  General  Services  Administration  It  further  recommended  that  the  DoD  pursue 
alternatives  to  reduce  requirements  for  skilled  personnel  in  calibration  and  repair  of  ETE 

The  support  of  older,  out-of-production  ETE  is  difficult  for  the  DoD  In  recognition  of 
this  difficulty,  the  task  .  ce  recommended  that  the  Dol)  systematically  replace  older  ETE,  using 
procedures  that  assure  budgeting  the  required  resources  and  recovery  of  the  value  of  the  old 


equipment  directly  to  the  ETE  programs 


To  increase  the  standardization  of  technical  documentation,  the  task  force  recommended 


that  the  DoD  establish  uniform  requirements  so  that  manufacturers  can  submit  commercial  ETE 
manuals  rather  than  military  manuals  without  need  for  revision.  It  also  recommended  that  the  DoD 
establish  uniform  provisioning  documentation  requirements  for  off-the-shelf  and  modified  ETE  and 
eliminate  the  requirement  for  this  documentation  except  when  military  need  dictates. 

Management 

In  the  area  of  ETE  management,  the  task  force  recommended  that  both  general-purpose 
and  special-purpose  ETE  be  placed  under  a  single  manager  within  each  Military  Service  A  minority 
recommended  that  those  single  managers  be  assigned  responsibility  for  reviewing  special-purpose 
ETE  purchases  and  assuring  that  off-the-shelf  ETE  could  not  satisfy  the  requirement,  but  that 
otherwise  management  of  special-purpose  ETE  be  separate. 

The  task  force  also  recommended  that  the  DoD  take  into  consideration  the  admin 
istrative  and  other  indirect  costs  associated  with  purchase  of  military  specification  ETE,  modified 
commercial  ETE,  and  off-the-shelf  ETE  under  prepriced  arrangements.  Those  costs  were  not  being 
considered  even  when  significant  differences  existed. 

Follow-Up 

The  task  force  identified  a  wide  range  of  problems  with  the  acquisition  and  management 
of  ETE  within  the  DoD  Those  problems  covered  the  gamut  of  acquisition  and  management  issues, 
from  the  use  of  military  specifications,  PILs,  and  commercial  support  capabilities  to  the  estab¬ 
lishment  of  a  single  manager  for  ETE  within  each  Military  Service  The  recommendations  of  the  task 
force  were  accepted  by  the  study  sponsors.  The  Assistant  Secretary  of  Defense  (Installations  and 
Logistics)  issued  a  memorandum  to  the  Assistant  Secretaries  of  the  Military  Departments,  dated 
28  April  1976,  announcing  establishment  of  a  program  to  monitor  implementation  of  the  task  force's 
recommendations.  The  task  force  was  extended  for  1  year  to  assist  in,  and  report  on,  implementation 


progress 


In  its  final  report,  the  task  force  noted  that  some  progress  had  been  accomplished. 


notably  in  the  areas  of  bid  sample  testing  and  standardization  of  commercial  manuals,  but  that  many 


of  the  recommended  actions  had  not  yet  progressed  beyond  the  "just  underway”2  stage.  In  response  to 
its  recommendation  to  provide  a  means  of  continuing  a  dialogue  between  the  Military  Services  and 
industry  on  ETE  acquisition  policy  and  procedures,  the  Deputy  Secretary  of  Defense  authorized 
formation  of  an  advisory  committee  and  requested  the  Army  to  sponsor  that  committee  as  an 
extension  of  its  lead  responsibility  for  standardization  of  Federal  Supply  Class  (FSC)  6625,  which 
consists  of  genera  1-purpose  ETE. 

Comments 

We  were  unable  to  obtain  additional  information  on  the  Assistant  Secretary  of  Defense 
(Installations  and  Logistics)  program  to  monitor  implementation  of  task  force  recommendations  after 
1977.  Moreover,  the  specific  long-term  results  of  the  task  force  study  are  difficult  to  assess. 

On  the  one  hand,  the  study  was  effective  in  opening  up  a  dialogue  between  the  Military 
Services  and  industry  (a  dialogue  that  still  continues  through  the  ETE  Division  of  the  American 
Defense  Preparedness  Association  and  its  annual  program  review),  in  increasing  the  awareness  of  the 
penalties  associated  with  unnecessary  military  specifications,  in  elevating  management  priority  on 
consolidation  of  metrology  and  calibration  facilities,  and  in  fostering  agreement  on  the  concept  of  "fly 
before  buy”  in  the  acquisition  of  ETE.  For  example,  the  Military  Services  have  recognized,  de  facto, 
the  benefits  of  using  commercial  off-the-shelf  test  equipment  rather  than  equipment  designed  to 
military  specifications  As  a  consequence,  they  are  procuring  proportionately  more  commercial  test 
equipment  than  in  the  past.  All  have  adopted  bid-sample  testing  as  a  standard  procedure  for 
competitive  procurement  of  commercial  test  equipment,  using  either  organic  testing  laboratories 
(Navy  and  Air  Force)  or  contractor  services  (Army)  (The  Army  is  planning  to  perform  its  own 
bid-sample  testing,  starting  in  fiscal  year  1985,  following  completion  of  an  advisory  study  by  the 
National  Bureau  of  Standards  about  the  appropriate  laboratory  equipment  and  testing  procedures  ) 

2The  Task  Force  on  Electronic  Test  Equipment,  "Implementation  Status:  Use  of  Off-the- 
Shelf  Electronic  Test  Equipment  to  Reduce  Costs,  Shorten  Leadtimes,  Assure  Reliability,  and 
Simplify  Logistics”  (Washington,  D  C  Defense  Science  Board,  Office  of  the  Director  of  Defense 
Research  and  Engineering,  January  1 977) 


have  not  been  implemented.  For  example,  current  procedures  (1)  do  not  exploit  commercial  support 
channels  and  warranties;  (2)  do  not  reflect  life  cycle  cost  factors  beyond  the  limited  "facility  of  use” 
factors  considered  in  competitive  bid  samples;  (3)  seldom  result  in  multiyear  procurement  contracts 
so  that  like-items  must  be  reprocured  year  after  year,  resulting  in  the  proliferation  of  different 
makes/models;  and  (4)  result  in  the  duplication  of  resources  committed  to  bid-sample  testing  and  test 
equipment  evaluation.  The  potential  savings  associated  with  these  four  items  alone  have  been 
estimated  to  range  upward  from  tens  of  millions  of  dollars  a  year 

In  sum,  most  of  the  task  force  recommendations  have  not  been  fully  implemented  by  the 
Military  Services  as  confirmed  by  the  OSD  effort  described  next 
DoD  AD  HOC  TMDE  STANDARDIZATION  WORKING  GROUP 


Noting  the  "numerous  studies  and  reports  concerning  commercial  off-the-shelf  ETE  and  the 
need  for  more  effective  standardization,”  the  Deputy  Under  Secretary  of  Defense  (Acquisition  Policy), 
DUSD  (Acquisition  Policy),  issued  in  May  1980  an  action  memorandum  for  the  Director,  Defense 
Material  Specifications  and  Standards  Office  (DMSSO)  That  memorandum  stated;3 

Failure  to  impose  a  viable  standardization  program  for  TMDE  severely 
impacts  maintenance,  manpower  resources  and  readiness  For  these 
reasons,  DMSSO  is  requested  to;  (1)  expedite  a  review  of  the  FSC  6625 
situation;  (2)  determine  and  take  action  to  accomplish  a  major  reduction  in 
the  existing  NSNs  (National  Stock  Numbers)  and  specifications  within  the 
FSC;  and  (3)  establish  a  more  effective  and  efficient  tri-Service  standard 
ization  program  to  avoid  future  uncontrolled  proliferation 

The  memorandum  further  authorized  DMSSO  to  establish  a  "DoD  Ad  Hoc  TMDE  Standard 
ization  Working  Group”  under  DMSSO  direction,  with  the  charter  to  "develop  a  comprehensive  DoD- 
wide  action  plan  to  improve  standardization  management,  methods  and  procedures  for  FSC  6625  ”  It 
also  required  that  the  standardization  program  plan  be  completed  in  1  year  and  that  program  tasks 


3DUSD  (Acquisition  Policy),  Memorandum  for  the  Director,  DMSSO,  Subject:  "Test,  Mea 
surement,  and  Diagnostic  Equipment  (TMDE),  FSC  6625,”  13  May  1980 


be  accomplished  within  a  3-year  period  "with  continuing  single-manager  operations  for  TMDE 
standardization.” 

Actions 

In  July  1981,  DMSSO  convened  the  first  meeting  of  the  working  group,  which  was  com¬ 
posed  of  Military  Service  representatives.  In  that  meeting,  the  working  group  determined  that  ten  of 
the  Defense  Science  Board  ETE  task  force  recommendations  pertaining  to  standardization  had  not 
been  fully  implemented.  It  also  made  two  decisions.  One,  it  decided  that  the  best  approach  for 
achieving  the  goals  of  the  DL'SD  (Acquisition  Policy)  was  through  a  Standardization  Program 
Analysis  and  Plan  in  accordance  with  the  Defense  Standardization  and  Specification  Program 
(DSSP).4  Second,  it  decided  to  conduct  an  item  reduction  study  for  FSC  6625,  using  the  methodology 
and  scope  developed  in  a  May  1981  meeting  of  DMSSO,  U  S.  Army  Communications-Electronics 
Command  (CECOM)  representatives,  Defense  Logistics  Agency,  and  Defense  Electronics  Supply 
Center  (DESC).  The  Army  (CECOM),  as  assignee  for  FSC  6625,  assumed  the  chair  of  the  working 
group,  with  meetings  planned  at  quarterly  intervals  to  clarify  and  monitor  the  item  reduction  study 
and  implementation  of  the  Defense  Science  Board  ETE  task  force  recommendations  pertaining  to 
standardization. 

In  October  1981,  the  first  meeting  of  the  item  reduction  study  team  took  place  at  DESC 
Military  Service  participants  discussed  differences  in  maintenance  concepts  for  TMDE,  basis  of 
selection  of  TMDE,  and  procedures  for  performing  item  reduction  studies.  DESC  noted  that  a 
working  group  method  (in  lieu  of  the  standard  procedures  of  the  DSSP)  was  necessary  because  of  the 
different  PILs  utilized  by  the  Military  Services  and  the  absence  of  interchangeability/substitution 
criteria  for  standard  item  selection.  The  participants  approved  the  initial  categories  of  TMDE 

4Office  of  the  Under  Secretary  of  Defense  (Research  and  Engineering),  "Defense  Standard 
ization  and  Specification  Program  Policies,  Procedures  and  Instructions,”  Defense  Standardization 
Manual  DoD  4120  3-M  [Washington,  D  C..  Office  of  the  Under  Secretary  of  Defense  (Research  and 
Engineering),  August  1 978 1 . 


selected  for  the  item  reduction  study  (frequency  counters,  function  generators,  pulse  generators, 
signal  generators,  and  multimeters). 

In  April  1982,  DESC  representatives  briefed  the  working  group  on  progress  accom¬ 
plished  to  date.  DESC  identified  132  distinct  families  (types  of  TMDE)  within  FSC  6625  These 
families  contained  approximately  13,360  makes  and  models,  from  which  an  estimated 
7,214  (54  percent)  would  be  eliminated  from  the  Federal  Supply  System  through  cancellation  or 
coding  the  NSN  "not  for  procurement”  ( i.e. ,  item  standardization  code  "3”  as  defined  in  Defense  Stan¬ 
dardization  Manual  DoD  4120.3-M).  DESC  reviewed  the  TMDE  categories  analyzed  to  date  and 
showed  a  schedule  for  item  reduction  studies  continuing  into  fiscal  year  1986  It  noted,  however,  that 
without  Military  Service  agreement  on  a  method  for  documenting  a  DoD  standard  item  of  TMDE, 
many  of  the  actions  requested  by  the  DUSD  (Acquisition  Policy)  could  not  be  accomplished. 

In  July  1982,  CECOM  released  its  first  report  on  progress  achieved  by  the  working 
group  in  accordance  with  DSSP  procedures. 5  It  issued  a  second  report  in  June  1984.<>  The  latter 
report  stated  that  12  generic  groups  of  TMDE  had  been  reviewed  for  item  reduction  as  of 
January  1984,  and  that  28  percent  of  the  NSNs  contained  in  those  groups  had  been  identified  for 
elimination  (1,428  of  a  total  of  5,1 18  NSNs).  The  report  provided  a  schedule  for  the  remaining 
FSC  6625  item  reduction  reviews  covering  19  families  of  TMDE  containing  7,756  NSNs  It  noted,  in 
spite  of  the  item  reduction  studies,  that  the  proliferation  of  TMDE,  as  measured  by  NSN  population, 
was  continuing  and  that  approximately  44  percent  of  the  NSN  population  consisted  of  "one-of-a-kind” 
makes  and  models  with  little  or  no  potential  for  elimination  through  formal  item  reduction  studies 
(see  Tables  2  1  and  2-2)  Further,  it  reviewed  the  status  of  the  Defense  Science  Board  ETE  task  force 
recommendations  and  the  problems  encountered  in  implementing  a  more  effective  standardization 
program  for  FSC  6625,  including  those  associated  with  cataloging  and  a  new  General  Services 

5Department  of  Defense,  "Standardization  Program  Analysis:  Electrical  and  Electronic  Prop 
erties  Measuring-Testing  Instruments,”  FSC  6625  (Washington,  DC  Department  of  Defense, 

1  July  1982) 

^Department  of  Defense,  "Standardization  Program  Analysis  Test,  Measurement  and  Diag¬ 
nostic  Equipment  (TMDE),”  FSC  6625  (Washington,  D  C  Department  of  Defense,  29  June  1 984) 


TABLE  2-1  FSC  6625  INVENTORY  TREND 


NSN  POPULATION'1 


YEAR 

Total 

DoD 

Army 

Navy 

Air 

Force 

Marine 

Corps 

Other 

DoD 

Dec  1978 

103,975 

83,811 

16,917 

26,805 

52,290 

4,548 

119 

Dec  1980 

108,202 

85,122 

17,334 

27,667 

53,570 

4,288 

187 

Dec  1982 

115,414 

87,286 

18,110 

29,302 

54,948 

4,081 

433 

Quantities  for  DoD  components  do  not  add  up  to  the  DoD  total  because  of  common  items. 


TABLE  2-2  FSC  6625  STANDARDIZATION  STATUS 
(December  1982) 


CATEGORY 

ISC1 

DESCRIPTION 

NSN  POPULATION 

Total 

With 

Manager 

Without 

Manager 

Procurable 

0 

DNA/NSA2  cognizance 

339 

314 

25 

1 

Standard  item 

434 

427 

7 

2 

Substitute  item 

7,711 

6,602 

1,109 

5 

Not  yet  subjected  to  IRS3 

42,279 

38,385 

3,894 

6 

One-of-a-kind  item 

50,521 

43,003 

7,518 

C 

IRS  completed,  no  decision 

12,748 

10,524 

2,224 

Nonprocurable 

3 

IRS  completed,  item  canceled 

1,589 

1,536 

53 

•ISC  =  Item  Standardization  Code 

2DNA  -  Defense  Nuclear  Agency;  NSA  =  National  Security  Agency. 
3IRS  =  Item  Reduction  Study 


Administration  policy  requiring  Navy  and  Army  usage  of  Federal  Supply  Schedules  in  procuring  FSC 
Group  66  items  (instruments  and  laboratory  equipment)  Finally,  the  report  concluded  the  following 

FSC  6625  standardization  issues  presented  in  the  Deputy  Under  Secretary 
of  Defense  (Acquisition  Policy)  Memo  dated  13  May  1980,  subject  TMDE, 

FSC  6625,  and  the  1975-1976  ETE  Task  Force-Defense  Science  Board 
recommendations,  frequently  have  been  addressed,  in  varying  degrees,  by 
each  military  service  rather  than  on  a  coordinated  tri-service  basis.  This  is 
due  to  the  fact  that  a  Tri-Service,  DoD  Ad  Hoc  Working  Group  has  never 
officially  convened  for  the  purpose  of  effectively  carrying  out  a  coordinated 
program  for  the  aforementioned  recommendations  and  policies. 

The  working  group  also  met  in  July  1984  to  review  progress  and  to  discuss  development 
of  a  DoD-wide  action  plan  with  a  new  charter  for  a  permanent  DoD  TMDE  Working  Group  At  that 
meeting,  the  working  group  decided  to  continue  the  planned  item  reduction  studies  and  to  complete  a 
Standardization  Program  Plan  in  accordance  with  the  DSSP.  A  first  draft  of  that  plan  was  scheduled 
for  completion  by  mid-1985.  In  its  September  1985  meeting,  the  working  group  deferred  promul 
gation  of  that  plan  until  early  1986.  It  also  decided  that  any  further  action  on  a  coordinated  program 
for  standardization  of  common  test  equipment  should  await  more  specif'*  tasking  by  the  Office  of  the 
Under  Secretary  of  Defense  (Research  and  Engineering)  It  is  currently  preparing  a  letter  to  request 
such  tasking 

Comments 

The  DoD  has  made  little  progress  toward  developing  and  implementing  a  DoD-wide  plan 
to  improve  ETE  standardization  management,  methods,  and  procedures  The  plan  is  scheduled  to  be 
completed  next  year,  10  years  after  the  Defense  Science  Board  ETE  task  force  report  and  6  years  after 
the  DUSD  (Acquisition  Policy)  action  memorandum  It  will  represent  the  first  major  effort  toward 
test  equipment  standardization  -  an  issue  that  has  surfaced  in  all  test  equipment  studies  dating 
back  to  the  early  I960’s.  In  the  meantime,  actions  by  the  Do  I)  Ad  Hoc  TMDE  Standardization 
Working  Group  have  been  limited  to  item  reduction  studies,  i.e,  the  elimination  of  military  specifi 
cations  for  obsolete  items  of  test  equipment. 

According  to  a  DMSSO  representative,  substantial  progress  toward  ETE  standard 
ization  will  not  be  forthcoming  until  the  Assistant  Secretary  of  Defense  (Manpower.  Installations. 
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and  Logistics),  ASD(MI&L),  becomes  an  active  participant  in  the  standardization  efforts.  Yet,  since 
late  1977,  the  ASD(MI&L)  has  not  been  involved  in  test  equipment  management  and  support  issues, 
other  than  peculiar  issues  addressed  in  the  weapons  system  acquisition  process  With  the  organi- 
zational  change  in  1984,  moving  the  Directorate  for  Weapons  Support  from  ASD(  MI&L)  to  the  Under 
Secretary  of  Defense  (Research  and  Engineering),  the  ASD(MI&L)’s  involvement  in  test  equipment 
management  has  ceased  completely.  However,  the  1985  reorganization,  combining  all  acquisition 
and  logistics  management  responsibilities  under  the  Assistant  Secretary  of  Defense  (Acquisition  and 
Logistics)  offers  the  opportunity  for  more  effective  action. 

ACQUISITION  IMPROVEMENT  PROGRAM  INITIATIVE  #30 

Studies  conducted  in  support  of  the  weapons  system  acquisition  improvement  program, 
initiated  by  the  Deputy  Secretary  of  Defense  in  1981,  identified  three  major  problems  in  the  program¬ 
ming  and  budgeting  for  system  support:7 

•  Support  program  and  budget  requirements  are  based  upon  standard  planning  factors  that 
may  not  match  readiness  objectives  of  a  new  weapons  system 

•  Development  and  fielding  of  a  weapons  system  involves  numerous  different  appro¬ 
priations,  while  the  budget  is  reviewed  by  appropriation  As  a  result,  budget  decisions  are 
often  made  in  isolation  without  visibility  of  the  impact  on  individual  system  support  and 
readiness 

•  Some  weapon  support  funds  are  controlled  by  functional  managers  not  responsible  to  the 
program  manager,  and  their  priorities  do  not  always  match  those  of  the  program  manager 
As  a  result,  budget  decisions  are  made  without  coordination  with  the  program  manager 
and  without  visibility  of  the  impact  on  individual  system  support  and  readiness. 

One  of  the  support  elements  affected  by  these  three  problems  was  support  and  test  equipment. 

Initiative  #30,  "Improved  Management  of  Support  Resources  for  Selected  Weapon  Systems,” 
was  designed  to  resolve  these  problems.  It  required  the  Military  Services  to  involve  the  program 
manager  in  the  development  of  support  resource  requirements  and  to  improve  the  visibility  of  those 
requirements  and  the  readiness  objectives  for  all  weapons  systems  entering  initial  production  at  any 
given  time.  It  also  required  OSD  to  conduct  a  single,  integrated  review  of  support  associated  with 

7Deputv  Secretary  of  Defense  Memorandum,  Subject:  "Improving  the  Acquisition  Process” 
(with  attachments),  30  April  1981 


individual  weapons  systems  at  key  decision  points  in  the  planning,  programming,  and  budgeting 
system  (PPBS)  process.  It  called  for  a  2-year  trial  period  for  the  implementing  procedures.  Respon¬ 
sibility  for  following  up  this  decision  was  assigned  to  the  ASDIMI&L).  A  joint  OSD/Military  P  rvice 
Steering  Group  and  a  working  group  were  established  to  oversee  implementation 
Actions 


In  the  1983  PPBS  cycle,  support  requirements  and  funding  information  for  23  weapons 
systems  entering  or  in  early  production  was  provided  at  key  decision  points.  OSD  reviewed  that 
information  and,  as  a  result,  raised  several  funding  issues  to  the  Defense  Resources  Board.  In  the 
1984  PPBS  cycle,  approximately  nine  weapons  systems  within  each  Military  Service  were  reviewed 
by  OSD.  A  description  of  the  procedures  and  some  of  the  problems  encountered  in  the  reviews  can  be 
found  in  a  Logistics  Management  Institute  report.8  In  late  1984,  the  Military  Services  submitted 
their  proposed  implementation  plans  in  accordance  with  OSD  guidelines.9  Following  OSD  concur 
rence,  they  will  put  those  plans  into  effect  with  the  1986  PPBS  cycle 
Comments 


The  procedures  tested  thus  far  have  been  successful  in  making  support  visible  in  the 
PPBS  by  weapons  system.  They  have  demonstrated  that  the  steps  required  to  identify  the  support 
requirements  and  funding  for  specific  weapons  systems  are  feasible  and  that  the  information 
identified  can  be  used  beneficially  by  the  OSD  to  influence  resource  allocation  decisions  made  during 
the  PPBS  cycle.  The  reviews,  however,  also  highlighted  the  need  for  a  disciplined  support  funding 
management  system  that  will  (1)  provide  credible,  validated  estimates  of  support  requirements  and 
funding;  (2)  assign  clear  accountability  for  changes  to  those  estimates;  and  (3)  track  changes  and 
their  effects  in  both  the  acquisition  system  and  the  PPBS 


8David  V  Glass  and  Donald  W  Srull.  Improved  Management  of  Support  Resources,  Final 


Support  Funding,”  28  August  1984 
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Standardization  and  improved  management  coordination  among  the  Military  Services,  both 
with  respect  to  acquisition  and  support  of  prime  equipment  as  well  as  support  equipment,  are  issues 
of  long-standing  interest  and  concern  to  the  Congress  and  the  General  Accounting  Office  (GAO)  Of 
the  numerous  congressionally  mandated  studies  on  those  two  issues,  three  appear  to  be  particularly 
relevant.  (1)  centralized  management  of  calibration,  (2)  increased  standardization  of  aviation  ground 
support  equipment  ( AGSE),  and  (3)  increased  standardization  of  avionics. 

Although  the  topic  of  the  third  study  includes  factors  other  than  test  equipment,  many  of  the 
factors  impeding  standardization  of  prime  equipment  also  apply  to  test  equipment.  Furthermore, 
prime  equipment  standardization  is  one  way  of  fostering  standardization  of  support  equipment 
CENTRALIZED  CALIBRATION  MANAGEMENT 


In  1977,  the  GAO  completed  a  study  of  the  calibration  systems  of  the  Military  Services  and 
Federal  agencies  It  found  that  each  operated  and  maintained  its  own  calibration  system,  with  the 
Military  Services  operating  over  700  calibration  facilities,  employing  9,000  civilian  and  military 
technicians,  and  making  about  3  million  calibrations  each  year,  using  calibration  equipment  valued 
at  $1.8  billion.  Adding  the  calibration  equipment  used  by  four  major  nonmilitary  Federal  users,  the 
GAO  estimated  the  total  cost  of  Federal  Government-owned  calibration  equipment  at  $2  7  billion 

In  examining  the  operations  and  workloads  of  the  various  organizations  responsible  for  plan 
ning  or  executing  the  calibration  programs  in  the  Military  Services,  the  GAO  noted  the  following  1 

•  Metrology  Centers  Each  of  the  three  Military  Services  operates  its  own  metrology  center 
for  technical  guidance  of  its  calibration  program,  including  development  of  calibration 
equipment  and  standards,  development  of  calibration  procedures,  establishment  of 
calibration  intervals,  and  specification  of  technical  training  requirements.  Only  the 
training  of  calibration  technicians  has  been  standardized  and  consolidated  The  centers 
use  different  criteria  in  setting  calibration  intervals  so  that  similar  pieces  of  equipment  are 


•United  States  General  Accounting  Office,  A  Central  Manager  Is  Needed  To  Coordinate  The 
Military  Diagnostic  and  Calibration  Program.  Letter  Report  to  the  Secretary  of  Defense.  LCD-77- 
427  (Washington,  DC  US  General  Accounting  Office,  31  May  1977) 


calibrated  at  different  intervals.  They  also  use  different  calibration  procedures  so  that 
identical  measurements  are  accomplished  using  different  calibration  standards, 
equipment,  and  manuals.  The  JLC's  Joint  Technical  Coordinating  Group  for  Metrology 
and  Calibration  (which  is  discussed  in  Chapter  4)  has  attempted  to  standardize  calibration 
procedures,  but  has  not  t  een  successful.  As  a  result,  the  GAO  notes,  "the  centers  continue 
to  triple  overhead  costs  for  preparing  separate  procedures  and  using  different  standards 
and  test  equipment  to  make  the  same  measurements  ” 

•  Primary  Standards  Laboratories.  The  Military  Services  operate  four  primary  standards 
laboratories  (the  N'avy  has  two,  one  on  each  coast),  employing  over  250  personnel  with 
facilities  and  equipment  worth  $33  million  (in  1976).  During  fiscal  year  1976,  those  labo¬ 
ratories  performed  approximately  26,000  calibrations  at  a  cost  of  over  $7  million  Though 
their  measurement  capabilities  are  quite  similar,  interservice  support  was  less  than 
5  percent  at  each  laboratory.  The  GAO  points  out  four  areas  of  inefficiency 

First,  workload  capacity  far  exceeds  the  requirement  at  each  laboratory  Three  of  the 
four  laboratories  individually  have  sufficient  capacity  to  support  the  combined 
workload  of  all  laboratories,  if  operated  on  three  shifts.  The  largest  of  the  four,  the 
Air  Force’s  measurement  standards  laboratory,  could  support  the  combined  workload 
on  a  single  shift. 

Second,  lower-level  facilities  often  send  their  test  equipment  or  calibration  standards 
to  their  own  Military  Service ’s  laboratory  rather  than  using  the  closest  laboratory. 

Third,  expensive  facilities  and  calibration  equipment  are  utilized  poorly 

Fourth,  the  four  laboratories  employ  indirect  labor  personnel  performing  duplicate 
functions 

•  Secondary,  Intermediate,  and  User  Calibration  Facilities  The  lower-level  calibration 
activities  use  "transfer  standards”  or  "shop  standards”  to  calibrate  the  test  equipment  of 
using  units  in  a  certain  geographic  area.  In  turn,  each  level  of  calibration  standard  is 
supported  by  a  higher-level  calibration  activity,  e  g  ,  shop  standards  from  intermediate 
calibration  activities  are  calibrated  on  transfer  standards  at  secondary  calibration 
facilities,  and  transfer  standards  are  calibrated  on  reference  standards  at  the  primary- 
standards  laboratories.  With  more  than  700 calibration  facilities  spread  around  the  world, 
many  regions  have-  multi}.  facilities  that  offer  the  potential  for  significant  savings 
through  consolidation  and  interservicing  Yet,  many  of  these  opportunities  for  consolida¬ 
tion  have  not  been  pursued  by  the  Joint  Technical  Coordinating  Group 

The  GAO  concluded  that  "DoD  continues  to  underutilize  its  resources  and  incurs  unnecessary- 
costs  for  transportation,  equipment,  staff,  and  facilities  "  It  recommended  that  the  Secretary  of 
Defense  "establish  a  single,  central  manager  for  the  entire  diagnostic  and  calibration  program.” 

Follow-L  p 


The  OSD  did  not  concur  with  the  GAO’s  recommendation  to  place  all  diagnostic  equip 
ment  under  a  single  manager  because  "the  technical  background  and  disciplines  involved  in  diag 
nostic  equipments  are  so  vast  that  any  attempt  to  piace  all  of  this  equipment  under  a  single  manager 


would  be  impractical  and,  in  fact,  counter  productive."2  However,  the  OSD  concurred,  in  principle, 
with  the  second  part  of  the  recommendation  because  "metrology  and  calibration  are  service  type 
functions  .  [thatl  do  lend  themselves  to  single,  central  management.  ’  The  OSD  did  point  out  that 
any  effort  to  consolidate  calibration  activities  must  be  carefully  planned  because  of  the  complexity 
and  magnitude  of  the  service  being  provided  by  those  activities.  Additionally,  it  stated  that  cali¬ 
bration  facilities  could  be  consolidated  provided  that  military  capabilities  are  not  adversely  affected 
and  that  wartime  surge  requirements  are  protected. 

Comments 

The  OSD  has  not  yet  implemented  the  GAO’s  recommendation  to  establish  a  single  man¬ 
ager  for  the  DoD’s  calibration  program.  Although  regional  consolidation  of  secondary/intermediate 
calibration  facilities  is  receiving  more  attention,  the  situation,  as  described  by  GAO,  has  changed 
little  in  the  past  8  years. 


STANDARDIZATION  OF  GROUN  D  SUPPORT  EQUIPMENT 

In  1978  and  1979,  the  GAO  examined  the  issue  of  standardized  ground  support  equipment  for 
military  aircraft  Its  report  noted  the  proliferation  of  like-type  equipment  in  the  DoD  inventory  and 
urged  that  actions  be  taken  to  increase  standardization,  thereby  reducing  acquisition  and  support 
costs  4  Some  examples  of  the  proliferation  found  by  the  GAO  are  shown  in  Table  3-1,  including 
selected  types  of  ETE  as  well  as  other  classes  of  support  equipment 

The  GAO’s  conclusions  and  recommendations  regarding  the  actions  that  should  be  taken  to 
control  proliferation  are  summarized  under  the  following  headings:  obstacles,  support  equipment 
planning,  data  systems,  review  process,  and  management  visibility 


-Letter  response  bv  the  Principal  Deputy  Assistant  Secretary  of  Defense  (Manpower,  Reserve 
Affairs,  and  Logistics),  Robert  B  Pirie,  Jr  ,  to  the  Director,  Logistics  and  Communications  Division. 
General  Accounting  Office,  Fred  J  Shafer,  dated  14  September  1977 

^Comptroller  General  of  the  United  States,  Increased  Standardization  Would  Reduce  Costs  of 
Grounu  Support  Equipment  for  Military  Aircraft,  LCD-80-30  (Washington.  DC  US.  General 
Accounting  Office,  7  February  1980) 


TABLE  3-1  EXAMPLES  OF  PROLIFERATION  OF  AIRCRAFT 
GROUND  SUPPORT  EQUIPMENT 


FEDERAL 
SUPPLY  CLASS 


ITEM  NAME 


NUMBER 
OF  DIFFERENT 
ITEMS 

IN  INVENTORY 
( 1979) 


1730 

(Aircraft  Ground  Servicing 
Equipment) 


Maintenance  platform 
Tow  bar,  aircraft 
Sling,  aircraft  maintenance 
Ladder,  aircraft  boarding 
Pin,  aircraft  ground  safety 
Cover,  aircraft  ground  servicing 
Shield,  aircraft  ground  servicing 
Jack  hydraulic  tripod 
Adapter,  hoisting 

Lock,  aircraft  ground  safety,  landing  gear 


4920 

(Aircraft  Maintenance 
and  Repair  Specialized 
Equipment) 


Power  supply 

Cable  assembly,  power  electrical 
Test  set,  fire  control  system 
Test  set,  (light  control  system 
Test  set,  radar 
Test  set,  indicator 
Test  set,  amplifier 
Maintenance  stand,  aircraft  engine 
Electronic  components  assembly 
Wiring  harness,  branched 


6625 

i Electrical  and  Electronic 
Properties  Measuring 
Testing  Instruments) 


Shunt,  instrument 

Oscilloscope 

Voltmeter,  electronic 

Cable  assembly ,  radio  frequency 

Lead,  test 

Ammeter 

Dolly,  test  equipment 
Transducer,  motional  pickup 
Indicator,  digital  display 
Galvanometer 


SOURCE  Comptroller  General  of  the  United  States.  Increased  Standardization  Would 
Reduce  Costs  of  Ground  Support  Equipment  for  Military  Aircraft.  LCD  80  30  (Washington.  I)  C 
U  S  General  Accounting  Office,  7  February  198(0 


Obstacles 


The  GAO  identified  four  obstacles  to  increased  standardization: 

•  Lack  of  integrated  support  planning  systems.  Weapons  system  acquisition  programs 
are  oriented  toward  meeting  a  single  aircraft’s  program  schedules  and  performance 
requirements  rather  than  providing  standard  support  equipment.  "Program  man¬ 
agers  are  hesitant  to  accept  the  risks  of  using  standard  equipment  on  any  new 
development  programs  if  the  contractor  recommends  new  equipment.” 

•  Lack  of  management  visibility.  The  Military  Services  possess  limited  visibility  and 
accounting  over  support  equipment  As  a  result,  equipment  that  may  never  be 
needed  is  bought  and  opportunities  for  more  equipment  commonality  are  missed 
"The  services  should  be  able  to  tell  the  contractor  what  equipment  they  prefer  rather 
than  routinely  accept  the  contractor’s  recommendations.” 

•  Lack  of  incentives.  According  to  DoD  contractors,  the  following  factors  impede  more 
standardization. 

Standardization  efforts  tend  to  have  an  unfavorable  image. 

Standardization  is  considered  a  constraint  against  technological  improvements. 

It  is  nearly  impossible  to  demonstrate  the  savings  that  are  derived  from  stan¬ 
dardization 

•  Lack  of  emphasis  on  early  support  equipment  planning.  The  Military  Services  as 
well  as  the  OSD  have  recognized  that  little  development  planning  for  support 
equipment  has  taken  place  because  such  planning  has  not  been  emphasized  With 
the  Military  Services,  support  equipment  requirements  are  not  identified  and 
screened  early  enough  in  a  weapons  system’s  life  cycle  to  achieve  commonality  or  to 
influence  system  design  to  match  existing  support  equipment  Within  the  OSD. 
there  is  no  focal  point  for  support  equipment  Such  a  focal  point  is  needed  for 
emphasizing  reduced  support  equipment  costs,  with  standardization  as  a  primary 
means 

To  overcome  these  obstacles,  the  GAO  recommended  that  the  Secretary  of  Defense 

•  Vigorously  pursue  a  policy  for  support  equipment  standardization 

•  Establish  a  focal  point  in  the  OSD  to  guide  and  direct  the  Military  Services  in 
carrying  out  such  policy 

•  Systematically  review  Military  Services'  activities  in  implementing  the  policy 

•  Develop  and  implement  incentives  for  contractors  to  use  existing  aircraft  support 
equipment  in  the  design  of  new  weapons  systems 

The  OSD  concurred,  in  principle,  with  three  of  the  recommendations,  but  objected  to  the 
second  recommendation  because  it  believed  that  DMSSO  should  provide  the  policy  direction  and 


guidance  for  implementing  ground  support  equipment  standardization.  (DMSSO  currently  is  respon¬ 
sible  for  providing  policy  and  guidance  on  standardization  of  all  DoD  materiel  items.)  In  addition,  the 
Navy  and  the  Air  Force  already  have  "focal  points”  for  support  equipment  standardization  and  have 
adopted  Joint  Service  standardization  programs  such  as  SISMS  (Standard  Integrated  Support 
Management  System).  Consequently,  there  is  no  reason  to  establish  another  special  DoD  focal  point 
for  standardization  of  AGSE.4 

Support  Equipment  Planning 

In  the  area  of  support  equipment  planning,  the  GAO  found  that  the  Military  Services 
have  not  been  participating  early  in  new  acquisition  programs  Although  the  Military  Services 
recognize  that  early  participation  in  the  aircraft  equipment  development  and  selection  process  is 
necessary,  such  participation  normally  does  not  take  place  until  the  aircraft  design  has  been  fully- 
developed  At  that  point,  design  changes  to  permit  more  standardized  support  equipment  are 
typically  impractical  and  not  cost  effective.  GAO  concluded 

The  services  need  to  formally  coordinate  efforts  of  weapon  system  program 
managers  and  support  equipment  managers  to  ensure  not  only  that 
schedules  and  aircraft  performance  requirements  are  met  but  also  that 
support  equipment  is  designed  and  intended  for  application  among  many 
aircraft  systems  where  practicable.  Controls  to  prevent  unnecessary  item 
proliferation  should  start  at  the  earliest  possible  stage,  that  is,  during  the 
design  of  new  aircraft  and  its  equipment 

The  GAO  recommended  that  the  Secretary  of  Defense: 

•  Develop  a  general  planning  strategy  for  support  equipment  that  not 
only  identifies  acquisition  problems  and  areas  for  increased  manage¬ 
ment  attention  but  also  takes  advantage  of  opportunities  to  promote 
standardization  and  reduce  the  number  of  different  support  equipment 
items 

•  Require  the  services  to  assess,  during  aircraft  design,  whether  support 
equipment  needs  can  be  satisfied  (11  by  using  the  existing  supply 
system  without  redesigning  the  aircraft,  (2)  by  altering  the  design  to 
accommodate  an  existing  piece  of  equipment,  or  (3)  if  new  equipment  is 
justified,  by  evaluating  whether  it  could  be  used  for  other  aircraft 


4Office  of  the  Secretary  of  Defense,  "Rationale  for  Non  Concurrence  with  Four  Recom 
mendations  in  Comptroller  Gen  ral  Report  No.  LCD-80-30  of  7  February  1980,”  enclosure  in  Letter 
Report  to  the  Comptroller  General.  10  April  1980 


•  Monitor  the  services’  planned  use  of  standard  support  equipment  items 
to  ensure  that  they  have  participated  in  the  equipment’s  design  and 
development  stages. 

The  OSD  did  not  accept  the  last  recommendation  because  it  believed  that  it  would  be 
extremely  difficult  for  DMSSO  to  monitor  the  planned  use  of  all  support  equipment  items  in  the  DoD 
However,  the  OSD  did  state  that  it  should  be  feasible  for  DMSSO  to  monitor  the  effectiveness  of  the 
DoD  departmental  standardization  offices  in  implementing  DoD  policies  and  guidelines  for  AGSE 
standardization. 

Data  Systems 

The  GAO  also  found  that  the  Military  Services  need  access  to  accurate  and  timely  data 
on  equipment  already  in  DoD  inventories  or  under  development  in  order  to  limit  the  introduction  of 
unnecessary  support  equipment.  The  data  systems  currently  in  use,  however,  contain  information 
that  is  inaccurate,  incomplete,  and  outdated  and  they  are  not  readily  accessible  Moreover,  cata¬ 
loging  problems  caused  by  the  use  of  unapproved  item  names  weaken  the  item  identification  process. 
The  overall  result  is  that  it  is  not  uncommon  for  more  than  one  NSN  to  be  assigned  to  the  same  item, 
thus  precluding  effective  control  over  unnecessary  proliferation.  The  three  FSCs  containing  most  of 
the  AGSE  include  approximately  222,450  NSNs,  but  53  percent  are  cataloged  without  approved  item 
names  The  GAO  recommended  that  the  Secretary  of  Defense: 

•  Evaluate  the  capabilities  of  the  various  support  equipment  data 
systems  to  determine  which  system(s)  can  most  promptly  provide  the 
most  complete,  up-to-date,  accurate,  and  readily  accessible  information 

•  Direct  the  services  to  include  all  necessary  support  equipment  items  in 
their  data  systems  The  systems  should  include  data  on  item 
description,  sizes,  shapes,  reliabilities,  capabilities,  and  applications 
The  systems  should  also  designate  preferred  items  that  the  Govern 
ment  wants  contractors  to  use  when  they  design  new  equipment 

•  Impose  tight  controls  where  new  items  enter  the  supply  system  so  that 
their  assigned  names  are  recognized  by  all  potential  users  when 
screening  available  data  systems  and  manuals 

The  OSD,  in  principle,  concurred  with  these  recommendations 
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Review  Process 


In  their  review  and  analysis  of  support  equipment  recommendation  data  (SERD), 
engineers  from  the  Military  Services  currently  base  their  decisions  largely  on  their  knowledge  of  and 
experience  with  the  general  functions  the  items  are  intended  to  perform.  No  concrete  methodology 
exists  to  logically  guide  the  engineers  through  the  review  process  so  that  they  can  decide  whether  the 
items  are  needed  and,  if  needed,  whether  a  standard  item  can  be  substituted.  For  a  sample  of  items 
studied,  the  GAO  found  that  very  little  documentation  was  available  to  justify  accepting  the  items. 

The  time  allowed  for  approving  or  disapproving  a  contractor-recommended  item  is  gen¬ 
erally  60  days,  both  for  initial  and  subsequent  SERD  submissions.  (A  SERD  is  submitted  for  each 
piece  of  support  equipment,  with  resubmissions  required  throughout  the  life  of  the  contract  when 
functional  requirements  change  or  previous  submissions  become  invalid.  A  contractor  who  does  not 
receive  a  response  within  75  days  from  SERD  submission  can  request  the  ordering  activity  to  issue  an 
order  for  the  support  equipment  item(s)  involved,  though  the  contractor  must  notify  the  authorizing 
activity  and  reviewing  activity  7  days  prior  to  the  effective  automatic  order  date.)5  As  a  result, 
reviewing  activities  are  under  great  time  constraints  and  often  must  delete  SERDs  "pending  evalua¬ 
tion”  if  they  anticipate  that  the  time  limit  will  be  exceeded  The  GAO  found  that  a  large  portion  of 
the  SERDs  it  examined  were  not  processed  within  the  specified  time  limit  and  concluded  that  the 
process  should  be  simplified  and  streamlined. 

The  GAO  recommended  that  the  Secretary  of  Defense. 

•  Develop  specific  methodology  to  guide  reviewers  through  the  review 
process  so  that  they  can  decide  realistically  whether  items  are 
necessary.  The  methodology  should  include  the  requirement  to  screen 
existing  inventories  and  justify  why  existing  assets  are  unacceptable. 


5See  the  "Standard  Integrated  Support  Management  System  (S1SMS),”  17  September  1982, 
which  is  a  joint  Air  Force/Army/Navy/Marine  Corps  publication  developed  in  the  late  I960’s  under 
auspices  of  the  JLC  for  multi-Service  aeronautical  programs  and  adopted  in  1972  by  the  Assistant 
Secretary  of  Defense  (Installations  and  Logistics)  as  part  of  its  long  range  plan  for  improving  the 
DoD  logistics  system.  The  SISMS  documentation  was  changed  in  1978  and  subsequent  years  to 
facilitate  broader  application  to  both  single-  and  multi-Service  systems  An  extract  of  SISMS  is 
included  in  Volume  III  of  this  report. 


•  Require  that  Air  Force  and  Navy  top  management  oversee  the  ade¬ 
quacy  of  the  review  process  and  take  an  active  part  in  the  approval  or 
denial  of  contractor-recommended  items.  [The  GAO  investigation  did 
not  extend  to  the  Army.  ] 

•  Clearly  define  the  review  roles  and  responsibilities  of  essential  organi¬ 
zations  and  eliminate  those  activities  which  provide  little  or  no 
substantive  assistance  in  deciding  the  adequacy  of  recommended  items. 

•  Decide  whether  the  time  c.nstraint  imposed  for  unusually  complex 
items,  such  as  avionics  testing  equipment,  is  appropriate.  If  not,  devise 
different  strategies  to  ensure  that  complex  equipment  can  be  carefully 
reviewed  and  delivered  when  needed. 

The  OSD  did  not  concur  with  the  last  recommendation  because  the  process  outlined  is 
consistent  with  the  need  to  ensure  timely  processing  of  support  equipment  recommendations  and  the 
deadline  simply  provides  a  checkpoint  for  review;  it  does  not  authorize  the  contractor  to  initiate 
"unapproved”  development  of  support  equipment. 

Management  Visibility 

The  GAO  also  found  that,  organizationally,  the  Military  Services  are  too  fragmented  to 
play  a  larger  role  in  initially  determining  what  support  equipment  items  they  need  for  new  aircraft 
They  have  not  established  a  centralized  activity  to  manage  support  equipment  and  its  acquisition 
The  various  organizations  involved  with  support  equipment  are  isolated  from  each  other  through 
either  breakdown  in  communications  or  philosophical  differences  In  many  instances,  support  equip¬ 
ment  users  fail  to  inform  weapons  system  and  support  equipment  managers  of  the  problems  encoun¬ 
tered  in  using  the  equipment 

Furthermore,  the  Military  Services  do  not  routinely  coordinate  their  support  equipment 
programs  when  they  introduce  new  aircraft  or  support  equipment  into  their  inventories;  nor  do  they 
have  a  systematic  method  for  evaluating  equipment  planned  or  in  use  by  the  other  Military  Services 

The  GAO  believes  that  by  taking  advantage  of  some  existing  in-house  capabilities,  such 
as  that  provided  by  the  Air  Force’s  Cataloging  and  Standardization  Office,  the  needed  visibility  over 
support  equipment  standardization  could  be  achieved  However,  the  role  of  that  office  in  standard 
ization  has  been  limited  because  of  the  lack  of  command  emphasis  and  trained  personnel  Many  of  its 
specifications  and  standards  are  outdated,  and  more  than  30  percent  of  them  have  not  been  reviewed 


within  the  last  5  years.  Furthermore,  the  standardization  program  analyses  of  the  various  FSCs 
comprising  support  equipment,  conducted  in  accordance  with  the  DSSP,  are  of  limited  value  because 
the  input  from  contributing  agencies  has  been  minimal. 

Representatives  from  the  Military  Services  frequently  identify  acquisition  regulations 
as  the  greatest  obstacle  to  increased  standardization.  Yet,  those  regulations  offer  several  opportu 
nities  to  foster  standardization,  including  multiyear  procurements  and  the  use  of  design  specifi 
cations  for  follow-on  procurements.  Additionally,  the  Secretary  of  each  Military  Department  is  au¬ 
thorized  to  negotiate  contracts  for  equipment  whose  standardization  and  interchangeability  of  parts 
are  necessary  in  the  public  interest  and  whose  procurement  through  negotiation  ensures  this. 

The  GAO  concluded  that  the  Military  Services  should  increase  their  management  visi¬ 
bility  over  the  entire  support  equipment  spectrum  to  advance  standardization  opportunities,  to  pre¬ 
clude  procuring  unneeded  equipment,  and  to  act  on  support  equipment  problems  at  user  levels  It 
recommended  that  the  Secretary  of  Defense: 

•  Establish  an  activity  to  coordinate  the  efforts  of  item  and  system 
managers,  equipment  specialists,  design  engineers,  users,  and  any 
other  group  participating  in  support  equipment  acquisition  The 
activity  should  maintain  management  visibility  over  support  equip¬ 
ment  and  assess  whether  more  standard  equipment  should  be 
developed 

•  Require  the  services  to  coordinate  their  research  and  development 
efforts  so  that  they  do  not  duplicate  support  equipment  items  per¬ 
forming  similar  functions,  particularly  for  aircraft  common  to  both 
services. 

•  Require  the  services  to  use  design  specifications  and  multiyear  procure¬ 
ments,  if  authorized  by  law  The  Secretary  should  provide  instructions 
for  the  services  to  use  negotiated  procurements  when  competitive 
means  to  increase  standardization  cannot  be  applied 

•  Direct  support  equipment  managers  to  coordinate  with  weapon  pro¬ 
gram  managers  and  field  activities  to  provide  and  get  feedback  to 
enhance  standardization 

•  Reinforce  the  services'  cataloging  and  standardization  organizations’ 
role  in  approving  new  equipment  by  requiring  their  input  before  such 
equipment  enters  the  system. 

The  OSD  did  not  fully  agree  with  the  first  recommendation  because  it  would  be  very 
difficult  for  an  activity  established  at  the  OSD  level  to  manage  selection  of  AGSE  for  the  Military 
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Services.  However,  it  did  state  that  a  study  will  be  conducted  to  determine  the  feasibility  of  estab¬ 
lishing  a  tri-Service  technical  advisory  organization  or  a  Defense  Logistics  Agency  technical  advisory 
activity  with  AGSE  standardization  responsibilities  similar  to  those  of  the  Military  Parts  Control 
Advisory  Groups. 

Follow  Up 

With  the  exceptions  noted  above,  the  OSD  agreed  with  several  of  the  GAO  recommenda 
tions.  It  also  established  a  "DoD  Ad  Hoc  Working  Group  for  Standardization  of  AGSE,”  chaired  by 
DMSSO,  to  review  the  recommendations  in  more  detail.  The  working  group  agreed,  on  14  February 
1980,  to  focus  on  the  following  seven  tasks. 

1.  Examination  and  evaluation  of  DoD  acquisition  policies  and  implementation  in¬ 
structions 

2.  Evaluation  of  AGSE  acquisition  management 

3  Evaluation  of  AGSE  data  storage  and  retrieval  systems 

4  Investigation  of  the  feasibility  of  establishing  an  AGSE  Technical  Advisory 
Group  for  assistance  to  the  Military  Services  and  contractors 

5  Investigation  of  the  acquisition  practices  utilized  by  commercial  airlines  for 
achieving  standardization  of  AGSE 

6.  Evaluation  of  Standardization  Program  Analyses  for  FSCs  1730,  1740,  4920,  and 
6625 

7.  Continuing  audit  by  DMSSO  of  the  implementation  of  AGSE  standardization 
policies  and  instructions  and  the  degree  of  inter-Service  and  interproject  stan¬ 
dardization  activity  and  achievement. 

The  working  group  also  decided  that  the  JLC  would  conduct  Tasks  1,  2,  3.  4,  and  6;  the  AGSE 
standardization  study  would  not  include  TMDE  (ESC  6625).  and  DMSSO  would  be  the  OSD  point  of 
contact  and  the  DoD  coordinating  office  for  the  standardization  study,  including  the  preparation  of 
the  OSD’s  response  to  GAO  at  its  conclusion  .•> 

6Office  of  the  Under  Secretary  of  Defense  (Research  and  Engineering)  and  Defense  Material 
Specifications  and  Standards  Office,  Memorandum  for  DoD  Ad  Hoc  Working  Group  for  Standardiza- 
tion  of  Aircraft  Ground  Support  Equipment  (AGSE),  Subject:  "Assignment  of  AGSE  Standard 
ization  Study  Tasks,”  16  July  1980. 


The  results  of  the  JLC  study  effort,  which  was  completed  in  1983,  are  reviewed  in 


Chapter  4.  We  have  no  information  on  the  planned  "final  OSD  report  to  GAO  ” 

STANDARDIZATION  OF  AVIONICS 

Congressional  interest  in  the  standardization  of  military  equipment  as  a  cost-saving  manage 
ment  principle  dates  back  to  1952,  when  the  Congress  enacted  the  Defense  Cataloging  and  Standard¬ 
ization  Act.  In  response  to  that  act,  the  OSD  issued  DoD  Directive  4120.3,  "Defense  Standardization 
and  Specification  Program”  (first  issue:  15  October  1954;  current  issue  10  February  1979)  The  ob¬ 
jective  of  the  DSSP  is  to  improve  readiness  and  assure  the  cost-effective  performance  of  equipment  by 
fostering  standardization  Responsibilities  for  establishing  standardization  policies,  procedures,  and 
controls  rest  with  DMSSO  and  with  the  Military  Services  responsible  for  the  implementation  and 
enforcement  of  those  policies,  procedures,  and  controls  Each  Military  Service  has  established  a 
departmental  standardization  office  to  manage  its  portion  of  the  standardization  program,  but  those 
offices  do  not  have  responsibility  for  specific  standardization  efforts 

The  standing  Congressional  committee  that  has  taken  a  particular  interest  in  DoD  standard¬ 
ization  is  the  House  Committee  on  Government  Operations,  which  has  both  legislative  and  oversight 
jurisdiction  for  "the  economy  and  efficiency  of  Government  operations  and  activities,  including 
Federal  procurement  ”  In  1970,  this  committee  found  that  "standardizing  even  fairly  simple  items  . 
was  extremely  difficult  in  the  face  of  differing  service  practices  and  preferences  and  manufacturing 
variations”  and  concluded  that  "  .  (in|  standardization,  as  in  so  many  other  important  but  neglected 

Government  functions,  sufficient  resources  are  not  made  available,  and  the  savings  potentials  are 
hardly  realized  ”7 

U nder  its  current  chairman,  Congressman  Jack  Brooks,  the  Committee  and  its  Legislation  and 
National  Security  Subcommittee  have  conducted  numerous  hearings  and  studies  to  monitor  the 


7Committee  on  Government  Operations,  Military  Supply  Systems  Cataloging.  Standardiza- 


DoD’s  progress  toward  standardization.  The  following  section  summarizes  the  Committee’s  most 
recent  report  to  the  Congress,  approved  and  adopted  by  the  Committee  on  31  July  1984.8 
Status  of  DoD  Standardization 

The  Committee  noted  that  "under  the  existing  organizational  relationships  and  lines  of 
responsibility,  standardization  initiatives  taken  within  DoD  have  often  failed  to  bring  results  ”  As 
supporting  evidence,  it  referenced  the  following  studies: 

•  The  Defense  Science  Board  Task  Force  on  Specifications  and  Standards  concluded  in 
its  report  (April  1977)  that  little  improvement  could  be  expected  until  senior  man¬ 
agement  became  more  actively  involved  with  program  details  rather  than  general 
directions. 

•  The  Board  recommended  strengthening  DoD  management  of  the  DSSP  Since  then, 
DoD  has  promulgated  various  initiatives  and  standardization  policies,  but  has  not 
enforced  them. 

•  The  Defense  Science  Board  Task  Force  on  Command  and  Control  Systems  Manage¬ 
ment  recommended  in  its  report  (July  1978)  a  central  organization  to  oversee  Joint 
Service  systems  to  ensure  compatibility  and  operational  effectiveness.  The  recom¬ 
mendation  has  not  been  implemented. 

•  In  its  report  "Management  of  the  Development  and  Procurement  of  Airborne  Elec¬ 
tronics  (Avionics)  by  the  DoD”  (May  1978),  the  GAO  recommended  that  the 
Secretary  of  Defense  accelerate  efforts  to  develop,  issue,  and  implement  a  standard 
ization  policy  for  avionics  and  other  electronics  and  then  monitor  the  development 
and  procurement.  After  issuing  the  policy,  the  OSD  did  not  take  any  action  to 
implement  it. 

•  The  Defense  Science  Board  1983  Summer  Study  on  Joint  Service  Acquisition  Pro¬ 
grams  (Briefing  Report,  August  1983)  found  lack  of  attention  to  Joint  Service 
standardization  to  be  a  problem  and  concluded  that  ad  hoc  management  is  not 
consistent  with  sound,  stable  programs  The  Board  recommended  a  formal  process 
be  instituted  to  ensure  the  necessary  level  of  Military  Service  involvement 

•  A  recent  OSD  contractor  study  (R  B  Toth  Associates.  "An  Assessment  of  the 
U  S  Defense  Standardization  and  Specification  Program,”  Final  Technical  Report 
for  the  Standardization  and  Acquisition  Support  Directorate,  1984)  found  that  many 
of  the  constraints  on  standardization  result  from  the  decentralization  of  standard¬ 
ization  responsibilities  and  the  prevalence  of  parochialism  among  the  Military 
Services  It  did  not  find  any  evidence  that  standardization  personnel  within  the 
Military  Services  were  following  the  objectives  set  bv  the  Defense  Materiel 


^Committee  on  Government  Operations,  Failure  To  Standardize  Avionics  Equipment  Has 
Cost  Millions.  Forty-First  Report  by  the  Committee  on  Government  Operations.  House 
Report 98-935,  98th Congress,  2nd  session  (Washington,  DC  :  U  S  Government  Printing  Office. 
2  August  1984) 


Specifications  and  Standards  Board  and  concluded  that  most  personnel  remained 
uninformed  of  those  objectives. 

Avionics  Standardization  Initiative 

In  1980,  the  DoD  recognized  the  potential  gains  from  standardization  by  establishing 
the  Joint  Services  Review  Committee  on  Avionics  Components  and  Subsystems  Standardization 
(JSRC).  The  JSRC  was  tasked  to  identify  opportunities  for  standardizing  avionics  subsystems,  to 
coordinate  efforts  in  developing  those  opportunities,  and  to  facilitate  progress  toward  agreed  objec¬ 
tives.  It  is  staffed  by  one  member,  on  a  part-time  basis,  from  each  Military  Service.  Funding,  admin¬ 
istration,  and  management  of  JSRC-sponsored  standard  subsystem  programs  is  the  responsibility  of 
the  appropriate  agencies  within  the  lead  Military  Service 

The  JSRC  decided  to  concentrate  its  efforts  on  core  avionics  (equipment  fulfilling  com¬ 
mon  aircraft  requirements  such  as  communications,  navigation,  and  attitude/heading  reference 
systems)  since  they  incorporate  mature,  stable,  and  low-risk  technology  and  their  standardization 
would  meet  with  less  resistance  from  the  Military  Services  than  would  that  of  mission  avionics 
(Moreover,  several  recent  development  programs  for  mission  avionics  show  increased  emphasis  on 
Joint  Service  sponsorship.  The  AN/ALQ-165  Airborne  Self  Protection  Jammer  program  for  Navy  and 
Air  Force  tactical  aircraft  and  the  Integrated  Electronic  Warfare  Systems,  which  is  a  joint 
Navy/Air  Force  program  are  just  two  examples  )  JSRC  members  proposed  over  30  candidate  sub¬ 
systems  and  selected  five  low-risk  items  for  initial  sponsorship  in  1981  standard  central  air  data 
computer,  digital  audio  distribution  system,  attitude  heading  reference  system,  data  transfer  loader/ 
verifier,  and  flight  data  recorder  9  In  1 983,  two  additional  candidates  were  selected  for  joint 
development  The  potential  cost  avoidance  for  producing  the  initial  five  subsystems  (compared  to 
nonstandard,  unique  subsystems)  was  conservatively  estimated  bv  the  JSRC  at  $770  million  (1983 
dollars) 

9For  further  details  on  these  subsystems  and  their  current  development  status,  refer  to  the 
Committee’s  report  or  the  C  S  General  Accounting  Office,  Increased  -Joint  Avionics  Standard 
ization  Could  Result  In  Major  Economies  And  Operational  Benefits,  Report  to  the  Secretary  of 
Defense,  NSIAD-84-127,  10  July  1984 


The  Committee’s  assessment  of  the  JSRC’s  progress  in  developing,  producing,  and 
fielding  standard  avionics  subsystems  was  that  "the  JSRC  has  failed  to  achieve  the  goals  set  for  it  ” 


The  Committee  attributed  that  failure  to  the  following  factors 

•  The  JSRC  is  a  low-level,  ad  hoc  management  committee  without  a  full-time  staff  and 
authority  either  to  resolve  requirements  and  disputes  among  the  Military  Services 
or  to  defend  its  programs  during  budget  reviews. 

•  The  JSRC  has  not  been  able  to  successfully  support  its  funding  requirements.  As  a 
consequence,  project  production  schedules  continue  to  slip  because  only  about  one- 
third  of  the  required  funds  were  budgeted.  Such  slippage,  in  turn,  causes  program 
managers  to  make  alternative  arrangements  to  protect  their  own  weapons  system 
programs. 

•  The  success  of  a  JSRC  project  requires  that  each  of  the  Military  Services  fully  fund 
its  share.  It  is  not  uncommon  for  only  the  Military  Service  with  lead  responsibility 
to  make  the  necessary  funds  available. 

•  Program  managers  are  under  no  obligation  to  use  Government-furnished  equipment 
jointly  developed  under  JSRC  auspices.  They  may  even  favor  alternative  avionics 
whose  funding  could  be  provided  under  the  same  program  element  as  JSRC- 
sponsored  projects. 

•  The  JSRC -sponsored  projects  appear  to  receive  official  command  support,  but  in 
actuality  do  not  In  some  cases,  JSRC  projects  have  not  received  the  desired  support 
because  of  procedural  problems.  In  September  1983,  DMSSO  attempted,  during  a 
major  systems  acquisition  review,  to  have  OSD  restore  $5  million  to  the  Army’s 
program  element  that  funds  its  share  of  JSRC-sponsored  projects  DMSSO’s  request 
was  rejected  because  DoD  procedures  allow  only  those  programs  costing  at  least 
$50  million  to  be  considered  in  OSD  reviews.  Both  the  Committee  and  GAO 
observed  that  "the  potential  for  $50  million  or  more  in  savings  offered  by  funding 
avionics  subsystem  development  is  not  accorded  the  same  priority  as  an  expenditure 
of  $50  million.” 

Conclusions  and  Recommendations 

The  following  is  quoted  verbatim  from  the  Committee’s  report 

VII  CONCLUSION 

A  substantial  part  of  the  Defense  budget  during  the  next  five  years  — 
amounting  to  $50  billion  -  will  be  spent  for  avionics  Since  most  aircraft 
require  numerous  avionics,  with  functions  that  vary  little  among  different 
aircraft  types,  an  effective  way  to  control  costs  is  to  develop  standard 
avionics  whenever  possible,  instead  of  purchasing  unique  avionics  for  each 
aircraft 

Unfortunately,  in  too  many  cases,  military  service  program  manag¬ 
ers  are  still  buying  unique  systems  and  components  to  fulfill  common 
generic  operational  requirements,  funds  for  the  development  and 
production  of  avionics  subsystems  are  directed  to  the  major  weapon  system 
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manager  rather  than  to  a  manager  who  is  -esponsible  for  standardization 
Also,  avionics  subsystems  selected  for  standardization  which  are  not 
committed  to  a  major  weapon  system  are  ranked  low  in  budget  priority  and, 
therefore,  either  fail  to  receive  any  funding  or  receive  significant  funding 
cuts. 

The  five  avionics  subsystems  discussed  in  this  report  offer  a  potential 
savings  of  $770  million.  This  is  only  a  small  percentage  of  the  cost 
avoidance  opportunities  that  are  available  through  standardization  of 
avionics  equipment.  However,  the  importance  placed  on  standardization 
varies  among  the  services,  thus  creating  diverse  degrees  of  support  and 
funding  Each  service  budgets  for  standardization  independently  and 
disburses  dollars  differently 

The  Committee  believes  that  standardization  efforts  are  likely  to  fail 
unless  adequate  funding  is  provided  to  develop  and  procure  these  items  of 
equipment  on  a  timely  basis.  The  key  is  placing  the  accountability  for 
standardization  programs  at  a  high  level  and  establishing  specific  budgets 
for  joint  development  programs. 

The  Committee’s  recommendations  and  their  dispositions  by  the  Secretary  of  Defense 
are  shown  in  Figure  3- 1 
Comments 


The  Committee’s  report,  while  scathing  in  its  assessment  of  standardization  within  the 
DoD,  appears  to  underestimate  the  potential  for  cost  avoidance  or  savings  through  a  sound  standard¬ 
ization  program 

The  Acquisition  Improvement  Program  initially  included  an  initiative  on  standardiza¬ 
tion  (Carlucci  Initiative  #21,  "Develop  and  Use  Standard  Operational  and  Support  Systems”),  but  the 
emphasis  has  apparently  changed  over  the  past  years.  Initiative  #21  was  not  included  in  the 
Consolidated  Acquisition  Improvement  Program  of  1 983 : 1 0  it  was  dropped  from  the  consolidated 
category —  "Improved  Support  and  Readiness”  — to  which  it  was  previously  allocated  1 1 

The  recent  changes  in  DoD  acquisition  policy  —  specifically  the  revisions  of  DoD 
Directive  5000  39,  "Acquisition  and  Management  of  Integrated  Logistic  Support  for  Systems  and 


*°Deputy  Secretary  of  Defense  Memorandum,  Subject:  "Priority  Defense  Management 
Initiatives,”  5  May  1983. 

11  Deputy  Secretary  of  Defense  Memorandum,  Subject  "Year-End  Report  on  the  Acquisition 
Improvement  Program,”  15  July  1982  (transmitting  the  Report  of  the  Acquisition  Improvement 
Steering  Group) 
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FIGURE  3-1  CONGRESSIONAL  MANDATE  FOR  AVIONICS  STANDARDIZATION 


DISPOSITION  OF  RECOMMENDATIONS  PRESENTED  IN 
FORTY-FIRST  REPORTOFTHE 
HOUSE  COMMITTEE  ON  GOVERNMENT  OPERATIONS 
2  AUGUST  1984 


It  has  been  recommended  that  the  Secretary  of  Defense  direct  the  Secretaries  of  the 
Army,  Navy,  and  Air  Force  to  take  three  actions: 

"( 1 )  Ensure  review  and  selection  of  candidate  avionics  subsystems  where  joint 
acquisition  would  be  appropriate,  and  assign  the  candidate  project  to  a  service  at  a  level 
high  enough  to  assure  adequate  support." 

-  By  memorandum  of  the  18  June  1984,  the  Under  Secretary  of  Defense,  Research 
and  Engineering,  requested  the  Joint  Logistics  Commanders  (JLC)  to  develop  a 
plan  for  implementation  of  the  Defense  Science  Board  1 983  Summer  Study 
recommendation  to  establish  a  formal  mechanism  for  review  and  selection  of 
joint  acquisitions  below  the  system  level  While  not  limited  to  avionics 
subsystems  and  components,  it  is  intended  that  a  formal,  institutionalized 
process  established  to  evaluate  requirements,  technology,  programs,  and  issues 
to  identify  properly  joint  Service  candidates  including  those  of  the  avionics 
community.  Inclusion  of  the  JSRC  as  an  element  of  the  JLC  management  review 
system  was  specifically  recommended.  It  is  anticipated  that  this  w  ill  result  m 
appropriate  lead  service  assignments  and  subsequent  management  support  to 
ensure  effective  development,  documentation,  and  use  of  standard  subsystems 
and  components, 

"(2)  Ensure  that  funding  levels  set  for  projects  sponsored  by  the  Joint  Services 
Review  Committee  on  Avionics  Components  and  Subsystems  Standardization  are 
sufficient  to  support  development  and  production  schedules  necessary  for  avionics  to  be 
available  as  Government-furnished  equipment  in  accordance  with  programmed 
installations  in  aircraft." 

-  By  separate  correspondence  the  Service  Secretaries  are  being  directed  to  set 
adequate  funding  levels.  Use  of  Government  furnished  equipment  is  a  separate 
matter  requiring  case-by-case  evaluation 

"(3)  Establish  a  dedicated  program  element  for  research  and  development  funds  as 
well  as  a  budget  line  item  for  procurement  funds  to  support  the  joint  avionics  program" 

By  separate  correspondence  the  Service  Secretaries  are  being  directed  lo 
establish  program  elements  for  core  avionics  projects  that  w  ill  be  separate  and 
distinct  from  other  development  and  procurement  efforts  This  should  enable 
OSD  and  the  Congress  to  identify  readily  such  projects  and  support  them 
accordingly,  while  retaining  the  ability  to  reprogram  funds  as  may  be  dictated  bv 
unforeseen  emergencies. 


SOURCE  Disposition  of  Recommendations  presented  in  Forty  First  Report  of  the 
House  Committee  on  Government  Operations,  2  August  1 984 
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Equipment”  (November  1983),  and  Military  Standard  (MIL-STD)  -1388-1  A,  "Logistic  Support  Analy¬ 
sis”  (April  1983)  —  are  clearly  designed  to  foster  standardization  when  practical  and  cost-effective. 
For  example,  the  new  MIL-STD-1388-1A  explicitly  includes  a  task  on  standardization  — Task  202 
Mission  Hardware,  Software,  and  Support  System  Standardization  This  policy  emphasis,  however, 
will  not,  by  itself,  achieve  maximum  standardization  of  test  equipment;  it  needs  to  be  supported  by  an 
aggressive  program  to  implement  the  policy  and  to  eliminate  the  factors  impeding  increased  stan¬ 


dardization 


4.  JOINT  SERVICE  EFFORTS 


This  chapter  reviews  the  activities  and  results  of  four  Joint  Service  efforts  to  improve  test 
equipment  management  and  support:  the  Industry/Joint  Services  Automatic  Test  Project,  the  JLC 
Panel  on  Automatic  Testing,  the  JLC  Panel  on  Standardization  of  AGSE,  and  the  Joint  Technical 
Coordinating  Group  for  Metrology  and  Calibration  (JTCG-METCAL).  The  latter  is  a  standing 
committee  of  the  JLC,  whereas  the  panels  are  working  groups  established  by  the  JLC  to  study  and 
report  on  particular  issues. 

1NDUSTRV/JOINT  SERVICES  AUTOMATIC  TEST  PROJECT 

The  Industry/Joint  Services  Automatic  Test  Project  was  an  outgrowth  of  the  Industry  Ad  Hoc 
ATE  Project  for  the  Navy  that  was  completed  in  April  1977  (see  description  in  Chapter  5)  Its 
objective  was  to  examine  management  and  technical  issues  involving  automatic  testing  It  was 
chartered  and  sponsored  by  five  industry  associations  and  directed  by  a  steering  committee  composed 
of  senior  executives  and  technical  experts  of  the  major  industrial  organizations  affiliated  with  the 
sponsoring  associations  in  close  collaboration  with  senior-level  advisors  from  the  Military  Services. 
The  project  was  organized  into  three  working  committees  (testing  technology,  acquisition  support, 
and  management)  that  coordinated  the  work  of  17  task  groups  Task  group  members  included 
275  experts  from  86  industrial  firms  and  11  colleges/universities,  supported  by  225  representatives 
from  the  Military  Services. 

The  task  groups  identified  problem  areas,  their  causes  and  impacts,  and  possible  solutions  A 
week  long  conference/workshop  provided  a  forum  for  proponent  and  adversary  positions  on  all  aspects 
of  DoD  use  of  automatic  testing  (Automatic  Test  Conference  and  Workshop,  San  Diego,  California, 
April  1978)  It  also  provided  the  data  that  the  task  groups  used  to  develop  1 10  specific  recom 
mendations.  During  1979,  those  recommendations  were  subjected  to  a  benefits  analysis  (largely 


qualitative)  and  integrated  into  a  cohesive  program  of  action  for  implementation.  The  project’s  final 
report,  published  in  1980,  represented  a  distillation  of  45  staff  years  of  study  and  analysis  1 
Summary 

Although  many  of  the  110  detailed  recommendations  in  the  project’s  final  report  pertain 
to  acquisition  and  testing  technology  issues,  the  recommendations  that  are  germane  to  test  equip¬ 
ment  management  and  support  are  summarized  in  the  Appendix  (Table  A- 1)  The  overall  thrust  of 
the  project  is  indicated  by  the  following  summary  recommendations  (assimilating  74  of  the  110  rec¬ 
ommendations  with  the  highest  ranking  based  on  projected  impact  on  readiness  and  life  cycle  cost) 
that  are  quoted  verbatim  from  the  final  report  in  order  of  priority. 

1  ORGANIZATIONS,  PEOPLE,  FUNDING 
Problem 

Despite  an  abundance  of  procedures,  directives,  specifications,  and 
other  documents  governing  maintenance  planning,  few  complex 
weapon  systems  have  been  deployed  with  an  adequate  support 
capability  This  paradox  has  its  origin  during  the  early  phases  of  the 
acquisition  process,  when  support  authority  is  most  needed  but  least 
effective,  largely  because  resources  are  too  easily  allocated  to  more 
immediate  problems. 

Recommendations 

Provide  for  a  policy  which  imposes  supportabilitv  requirements 
for  acquisition  of  military  systems,  starting  at  the  conceptual 
phase. 

Implement  the  above  policy  at  the  individual  Service  level  bv 
establishing  centralized  organizations  with  appropriate  account¬ 
ability,  budget  control,  and  responsibility  for  interservice  coordi¬ 
nation. 

Provide  career  paths  and  motivation  for  retention  of  manage¬ 
ment  and  critical  technical  personnel ,  military  and  civilian 

Payback 

Potential  annual  savings  of  $100  million  in  support  equipment 
acquisition  costs. 

Across-the-board  improvement  in  military-equipment  avail¬ 
ability  for  major  systems  now  under  development 


1  Industrv/Joint  Services  Automatic  Test  Project:  Final  Report  (Washington.  D  C  Industry/ 
•Joint  Services,  June  1980). 
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MILITARY  EQUIPMENT  DESIGN 


Problem 

The  ability  to  test  military  equipment  efficiently  is  prerequisite  to 
supportability.  Testability,  as  a  design  discipline,  is  currently  inade¬ 
quate  because  there  is  no  accepted  method  for  measuring  it  and  no 
mechanism  for  imposing  and  enforcing  it  during  the  equipment 
design  phase. 

Recommendations 

Develop  verifiable  testability  requirements. 

Impose  these  requirements  on  the  prime-system/automatic-test- 
system  design  process,  and  take  measures  to  ensure  compliance. 

Payback 

A  potential  30  percent  improvement  in  system  availability. 

Significant  reductions  in  the  cost  of  support  system  hardware 
and  software. 

SPECIFICATIONS,  DIRECTIVES,  CONTROLS,  DELIVERABLES 
Problem 

Logistic-support  directives,  specifications,  and  standards  are  not 
applied  uniformly  or  early  enough  in  the  acquisition  process. 
Contract  Data  Requirements  items  are  redundant  and  duplicative 
across  the  Services,  and  have  proliferated  to  satisfy  individual 
requirements. 

Recommendations 

Impose  standardized  Contract  Data  and  Automatic  Test  Require¬ 
ments  documentation  as  program  planning  and  life-cycle  cost 
control  deliverables. 

Require  appropriately  tailored  versions  of  Logistic  Support 
Analysis/Logistic  Support  Analysis  Record  (LSA/LSAR)  develop¬ 
ment  procedures  early  in  weapon-system  acquisition. 

Payback 

Increased  interchange  of  data  among  the  Services 

Significant  reductions  in  the  cost  of  logistic  support  hardware 
and  software  across  the  Services 

Program  management  better  able  to  benefit  from  lessons 
learned 


NONELECTRONIC  TEST  DEVELOPMENT 


Problem 

Lack  of  effective  automated  maintenance  equipment  currently 
results  in  degraded  equipment  availability,  and  excessive  turn¬ 
around  times  and  costs  associated  with  logistic  spare-parts  recla¬ 
mation 

Recommendation 

-  Accelerate  the  application  of  automatic  test  in  support  of  non¬ 
electronic  systems  and  equipment.  Technology  is  available 
which  can  significantly  improve  readiness  and  fuel  efficiency, 
and  significantly  reduce  life-cycle  cost  and  maintenance  man¬ 
hour  requirements. 

Payback 

A  potential  30  percent  reduction  in  maintenance  man-hours  per 
operating  hour 

A  potential  20  percent  reduction  in  the  cost  of  spares. 

A  potential  10  percent  reduction  in  the  fuel  consumption  of 
internal-combustion-engine-powered  equipment 

TEST  PROGRAM  SET  DEVELOPMENT  AND  MANAGEMENT 

Problem 

Although  Test  Program  Set  costs  exceed  hardware  costs,  they  are 
less  predictable  and  less  controllable  Moreover,  there  is  no  common 
definition  of  what  constitutes  a  Test  Program  Set  among  designers, 
suppliers,  and  Government  agencies  As  a  consequence,  the  user 
receives  a  different  support  and  maintenance  data  package  of  vary¬ 
ing  quality  with  each  automatic-test  acquisition 

Recommendations 

Define  and  establish  controls  for  acquisition  and  maintenance  of 
Test  Program  Sets,  including  test  software,  interface  hardware, 
and  data 

Support  the  development  of  automated  test-program  generation 
systems 

Support  ATLAS  (Abbreviated  Test  Language  for  All  Systemsl  as 
the  common  Joint  Services  test  language. 

Payback 


Reductions  in  the  cost  of  Test  Program  Sets,  which  is  the  major 
cost  factor  associated  with  automatic  test 


Predictable,  measurable,  and  therefore  manageable  Test  Pro¬ 
gram  Set  development. 


AUTOMATIC  TEST  TECHNOLOGY  DEVELOPMENT 
Problem 

New-technology  devices  in  developmental  weapon  systems  pose  test 
problems  that  cannot  be  solved  using  traditional  test  techniques. 
Support-equipment  developers  need  advanced  test  techniques  and 
advanced  automatic-test  system  architectures  appropriate  to  these 
increasingly  complex  test  requirements. 

Recommendations 

Establish  continuing  technology-development  programs  in  spe¬ 
cific  aspects  of  automated  test  where  the  payback  potential  is 
high 

-  Support  a  technology-forecasting  activity  for  timely  identifica¬ 
tion  of  technology  advances  destined  to  impose  new  automatic- 
test  requirements  or  to  enhance  automatic-test  capabilities. 

Payback 

A  potential  technological  breakthrough  capable  of  dramatically 
reducing  the  costs  associated  with  support  hardware  and  soft¬ 
ware. 

DATA  BANKS  AND  MODELS  FOR  LIFE-CYCLE  COSTING, 
LOGISTIC  SUPPORT  ANALYSIS,  AND  TECHNOLOGY 
ASSESSMENT 

Problem 

Within  the  Department  of  Defense,  there  are  many  similar  but 
separate  data  banks,  each  with  its  own  access  procedures  The 
diversity  of  models  and  their  implementation  further  weakens  the 
integrity  of  the  systems  engineering  process  The  result  is  a  less 
than  optimum  interchange  of  Joint  Services  and  industrial  informa¬ 
tion. 

Recommendations 

Establish  common  models  and  Logistic  Support  Analysis 
techniques  tailored  to  the  systems  engineering  process  during 
various  phases  of  acquisition. 

-  Establish  a  linking  data-bank  network  to  improve  data  common¬ 
alty  and  the  ability  to  use  lessons  learned  across  the  Services. 

Payback 

Improved  management  understanding  of  the  true  costs  of  auto¬ 
mated  test,  as  differentiated  from  those  of  prime  systems 


Improved  tools  for  effective  prediction,  measurement,  and  con¬ 
trol  of  support  costs. 

Utilization  of  lessons  learned  across  Service  lines. 


8.  SYSTEM-SOFTWARE  DEVELOPMENT  AND  MAINTENANCE 
Problem 

The  development  of  automatic-test  system  software  is  a  complex 
process,  which  differs  subtly  from  that  for  other  DoD  software 
Unfortunately,  there  is  no  consistent,  top-down  understanding  of  the 
complex  hardware/software  relationships  involved.  As  a  conse¬ 
quence,  cost  reduction  and  control  are  ineffective,  and  software 
maintenance  is  unnecessarily  hampered  by  the  many  versions  of 
nonrehostabie,  proprietary  software  products  that  are  developed. 

Recommendations 

Rigorously  define  software  life  cycle,  and  requirements  for  con¬ 
figuration  control  and  quality  assurance. 

Develop  guidelines  for  configuration  management  and  for  the 
maintenance  of  automatic-test  system  software. 

Payback 

Significant  reductions  in  software-development  costs,  the  result 
of  managing  the  true  cost  drivers. 

A  20  percent  reduction  in  software  maintenance  costs. 

Option  for  software  organic  maintenance  across  the  Services. 

9  METROLOGY  AND  CALIBRATION 

Problem 

The  establishment  of  test-system  tolerances  and  accuracy  require¬ 
ments  is  hampered  by  the  absence  of  technical  criteria  and 
discipline.  Typically,  test  system  specifications  are  characterized  in 
terms  of  component  measurement  and  stimulus  units.  Few  auto¬ 
matic  test  systems  have  been  specified  to  a  common  reference  point, 
or  take  into  account  the  effects  of  interfaces  and  adapters  The  conse¬ 
quences:  trial-and-error  software  changes,  arbitrary  accuracy 
derating,  and  unnecessary  removal  of  units  for  calibration. 

Recommendations 

More  actively  involve  Metrology/Calibration  Centers  at  an  early 
stage  in  automatic  test  design  and  support  functions. 

Involve  the  National  Bureau  of  Standards  in  basic  measurement 
standards,  and  support  research  in  technique  development  for  in 
place  traceability 
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Payback 


Greater  consistency  in  the  quality  of  measurements  made  in  the 
field  and  at  depots. 

Improved  weapon-system  and  support-equipment  availability 
and  reduced  on-site  spares  requirements 

Fewer  man-hours  dedicated  to  test-equipment  calibration. 

10  TRAINING 
Problem 

The  effective  skill  levels  of  the  automatic-test  operator  and  mainte¬ 
nance  technician  are  increasingly  overwhelmed  by  the  requirements 
of  contemporary  technology.  Supervisor  training  is  particularly 
weak  and,  aggravated  by  lack  of  motivation,  results  in  a  high  rate  of 
turnover  among  the  better  people.  Training  still  relates  to  basic 
skills  and  traditional  training  methods,  creating  a  mismatch 
between  the  instructional  methodologies  used  and  the  highly 
sophisticated  equipment  involved. 

Recommendations 

Plan,  formalize,  develop,  and  fund  innovative  approaches  to  the 
training  of  support-equipment  operators,  maintenance  techni 
cians,  and  shop  supervisors 

Establish  formal  training  courses  for  personnel  at  all  levels  - 
acquisition  managers,  engineers,  and  technicians 

Payback 

Improved  motivation,  retention,  and  time  in  service  for  techni¬ 
cians/supervisors 

Increased  individual  and  unit  productivity 

Improved  weapon-system  and  support-equipment  availability 

11  MAINTENANCE  SHOP  PRODUCTIVITY 
Problem 

The  Maintenance  Shop  Supervisor  is  hampered  in  his  management 
of  support-equipment  resources  by  the  absence  of  real-time  data  on 
such  items  as  status,  priority,  production,  manning,  and  inventory: 
automated  processes  cannot  be  efficiently  managed  through  notes 
made  on  the  back  of  an  envelop  with  a  stubby  pencil 

Moreover,  contemporary  automatic  test  equipment  requires  a  con¬ 
trolled  environment  and  a  stable  power  source  for  proper  operation. 
It  contributes  an  excessive  level  of  added  acoustic  noise  to  the  work 
area,  and  currently  suffers  from  excessive  downtime  for  calibration 
and  repair,  as  well  as  from  too  many  and  too  complex  interface 
devices 
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Recommendations 


Improve  support-equipment  design  and  system  performance  by 
better  integrating  such  specific  Integrated  Logistic  Support 
elements  as  reliability,  maintainability,  and  human  factors. 

Establish  career  paths  and  provide  adequate  incentives  to  ensure 
retention  of  automatic-test  trained  and  experienced  personnel, 
both  military  and  civilian. 

-  Develop  and  implement  a  real-time  management  information 
function  under  local  control,  to  service  the  larger,  automatic- 
test-equipped  maintenance  shops  and  to  monitor  productivity. 

Integrate  facility  environmental  needs  into  shop-site  planning. 
Payback 

Improved  Maintenance  Shop  productivity  in  the  form  of  more 
rapid  turn-around  and  consistently  higher  repair  quality 

Intermediate-level  shops  better  able  to  handle  surge  workloads 
in  the  forward  area. 


Follow-Up 

The  project  had  extensive  visibility  within  the  DoD.  The  final  report  received  wide 
distribution  and  attention  and  increased  the  awareness  within  industry  and  the  DoD  of  the  nature 
and  extent  of  automatic  test  problems.  Some  of  the  recommendations  have  been  reflected  in  changes 
in  DoD  acquisition  policy  (particularly  ie  5000  series  of  directives/instructions)  and  military  stan¬ 
dards  Most  of  the  project’s  recommendations  were  adopted  by  the  JLC  Panel  on  Automatic  Testing 
for  further  study,  resolution,  and  action. 

Furthermore,  one  of  the  sponsoring  organizations,  the  National  Security  Industrial 
Association  (NSIA),  established  an  Automatic  Testing  Working  Group  (restructured  into  a  formal 
NSIA  Committee  in  1982)  to  continue  the  dialogue  between  industry  and  Government  and  to  support 
the  JLC  Panel  on  Automatic  Testing  The  activities  of  the  NSIA  Automatic  Testing  Committee 
include  sponsorship  of  training  courses,  organization  of  conferences,  liaison  with  the  Military 
Services,  and  a  number  of  special  study  projects  such  as  standardization,  integrated  diagnostics,  and 


artificial  intelligence.  Current  action  items  for  this  committee,  resulting  from  the  1984  ATE  Confer¬ 
ence,  "Supporting  Weapon  System  Technology  Through  the  1990’s,”  include  the  following: 

•  Methods  for  implementing  required  incentives  and  warranties  to  ensure  supportable 
weapons  systems  (Policy  and  Acquisition  Subcommittee) 

•  Follow-up  workshop  on  new  technology,  including  computer  aided  design/testability 
integration,  modeling  tools,  and  testing  techniques  (New  Testing  Technology  Sub¬ 
committee) 

•  Technical  input  to  the  JLC  Panel  on  Automatic  Testing  in  developing  common  stan 
dards,  guides,  and  tools  for  standard  ATE  applications  (Test  Software/Testability/ 
Automatic  Test  Systems  Subcommittees) 

•  Means  for  emphasizing  the  need  for  increased  investment  in  technologies  for  testing 
nonelectronic  equipment  (Mechanical  System  Condition  Monitoring  Subcommittee) 

•  Methods  for  improving  the  support  of  existing  weapons  systems  (Weapons  system 
Technology  Subcommittee) 

•  Design  tools  needed  for  built-in  test  (BIT)  design  in  very-high-speed  integrated  cir¬ 
cuits  applications  (Testability  Subcommittee) 

•  Methods  for  improving  communications  with  the  Congress  and  the  public  on  the 
need  for  improving  supportability  (Communications/Education  Subcommittee). 

JLC  PANEL  ON  AUTOMATIC  TESTING 

At  the  request  of  a  group  of  military  advisors  involved  in  the  Industry/Joint  Services  Auto 
matic  Test  Project,  the  JLC  chartered  a  Panel  on  Automatic  Testing  in  March  1978  and  approved  its 
study  plan  in  October  1978  The  Panel’s  mission  was  stated  as  follows:2 

to  develop  and  implement  a  long-range,  definitive  action  program  on 
automatic  testing.  The  panel  has  identified  over  80  closely  related  and 
interwoven  tasks  to  be  performed  during  the  next  five  years.  We  believe 
significant  savings  in  manpower  and  funds,  along  with  improved  opera¬ 
tional  readiness,  can  be  realized  from  these  efforts. 

The  panel  is  composed  of  two  members  from  each  of  the  four  participating  logistics  commands 
(Army  Materiel  Command  (AMC),  Naval  Material  Command  (NAVMAT),  Air  Force  Logistics  Com¬ 
mand  (AFLC),  and  Air  Force  Systems  Command  (AFSC)l,  plus  associate  members  representing  the 
U  S  Marine  Corps  and  the  Defense  Logistics  Agency  The  Panel  meets  quarterly  to  review  progress 


2”Joint  Agreement  on  Support  of  the  Automatic  Testing  Program,’’  signed  by  the  JLC, 
16  July  1979 


and  identify  new  task  requirements  and  conducts  an  annual  program  review  to  assess  the  overall 
program  and  to  update  the  Subtasks  Descriptions,  which  identifies  all  subtasks  in  a  standard  format, 
including  office  of  primary  responsibility,  milestones,  and  funding. 

Summary  and  Current  Status 

The  Panel’s  program,  consisting  of  approximately  80  tasks,  covers  three  broad  areas: 

•  Management:  Development  of  policies,  procedures,  guidance,  and  controls  needed 
for  weapons  system  acquisition  program  managers  to  integrate  automatic  testing 
into  the  acquisition  process. 

•  Acquisition  Support:  Development  of  tools  to  improve  procurement  techniques  for 
automatic  testing,  including  design  guides,  standards,  specifications,  and  training 
programs 

•  Testing  Technology:  Research  and  development  tasks  to  advance  the  state-of-the- 
art  in  automatic  testing  technology,  including  software  tools  and  languages, 
testability  concepts  and  techniques,  BIT  design,  and  new  technology  applications. 

A  more  detailed  listing  of  the  tasks  in  the  management  and  acquisition  support  areas,  excerpted  from 

the  1982  Subtasks  Descriptions.3  is  provided  in  the  Appendix  (see  Table  A-2)  Some  of  the  tasks  have 

been  completed,  others  have  been  deleted,  and  many  are  still  in  progress  Those  tasks  that  have  been 

completed  have  resulted  in  publication  of  several  Joint  Service  guides  (see  Figure  4-1),  conduct  of 

training  courses,  publication  of  a  quarterly  ATE  newsletter,  advances  in  testing  technology,  and 

issuance  of  improved  policies  and  procedures 

In  1983,  the  structure  of  the  program  was  revised  into  seven  functional  subgroups, 
policy  and  procedures,  test  program  sets  (TPSs),  testability,  new  technology,  off  line  ATE, 
communications/education,  and  machinery  testing.  With  the  1984  annual  program  review,  con¬ 
ducted  30  October  through  1  November  1984,  the  number  of  active  subtasks  declined  to  52,  divided 
among  the  seven  functional  subgroups.  With  most  of  the  original  subtasks  completed  in  1985,  the 
Joint  Secretariat  recommended  that  the  Panel  be  terminated  in  1985  The  Panel’s  membership, 
however,  is  in  eement  on  the  need  for  continued  Joint  Service  coordination  on  automatic  testing 


3JLC  Panel  on  Automatic  Testing,  Suhtask  Descriptions  (Washington,  DC  Joint  Logistics 
Commanders,  30  September  1982) 
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FIGURE  4-1.  SAMPLE  PRODUCTS  OF  THE  JLC  PANEL 
ON  AUTOMATIC  TESTING 


Joint  Service  Weapon  System  Acquisition  Review  Guidelines  for 
Automatic  Testing 

Joint  Service  Automatic  Testing  Acquisition  Planning  Guide 
Built-in-Test  Design  Guide 

Selection  Guide  for  Digital  Test  Program  Generation  Systems 

MIL-STD-2165,  Testability  Program  for  Electronic  Systems  and 
Equipments,  January  1985 

MIL-HDBK^XXX,  Testability  Analysis  Handbook  (Revised 
Draft,  March  1985 ) 


1  Military  Handbook. 


issues  and  dialogue  with  industry  as  technology  advances.  Consequently,  the  Panel  is  currently  in  a 
transition  stage.  The  most  recent  annual  program  review,  conducted  19  through  21  November  1985, 
is  expected  to  result  in  establishment  of  a  "Joint-Service  Automatic  Testing  Review  Board”  as  a 
successor  to  the  Panel,  with  a  broadened  charter  that  includes  the  following  responsibilities  (in 
accordance  with  the  draft  Joint  Service  regulation  that  has  not  yet  been  approved) 

•  Develop  methods  for  reducing  hardware,  software,  and  manpower  costs  associated 
with  automatic  testing 

•  Design  policies,  plans,  and  procedures  in  the  use  of  automatic  testing  hardware  and 
software  to  impro\  operational  readiness  of  weapons  systems. 

•  Facilitate  exchange  among  the  Services  and  OSD  of  technical,  managerial,  and  oper 
ational  information  on  automatic  testing  hardware  and  software  as  it  applies  to  the 
support  of  weapons  systems 

•  Manage  the  automatic  test  technology  standards  program  as  the  DoD  designated 
agent  under  DMSSO 

Comments 


The  JLC  Panel  on  Automatic  Testing  has  fostered  improved  communication  among  the 
testing  community  and  has  increased  the  visibility  of,  and  expertise  in,  testing  issues  in  the  acquisi 
tion  process  Through  the  products  identified  in  Figure  4- 1  and  the  presentation  of  various  training 
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courses,  the  Panel  has  had  some  impact  on  the  management  and  support  of  automatic  testing,  but 
most  of  its  program  has  focused  on  testing  technology  The  Panel  claims  that  "the  financial  rewards 
have  already  shown  that  the  cost  avoidance  of  the  present  program  equals  the  entire  budget  of  the 
Panel’s  program  by  eliminating  duplication  of  effort  and  development  of  a  Joint  Service  approach  to 
AT  [automatic  test |  systems  design  and  acquisition.”4 

A  review  of  the  funding  profile  of  the  subtasks  identified  in  the  1982  Subtask  Descrip¬ 
tions  shows  that  the  program  is  primarily  focused  on  testing  technology:  50  percent  of  the  tasks  and 
over  80  percent  of  the  funds  are  devoted  to  testing  technology  (see  Table  4-1).  The  top  seven  tasks, 
accounting  for  $260  million  of  the  total  budget  of  $320  million,  are  in  testing  technology  and 
concerned  with  the  following  purposes: 

•  Advanced  ATE  Concepts:  Air  Force  family  of  ATE  and  Navy  family  of  ATE 
($128  million  and  $80  million,  respectively) 

•  Microwave  ATE  ($7.5  million) 

•  Self-Improving  Diagnostics  ($6.2  million) 

•  Fiber  Optic  Technology  *$18. 2  million) 

•  Vehicle  Field  Testing  Systems  ($16.5  million) 

•  Unit-Under-Test  Simulator  ($5  7  million) 

In  contrast,  the  subtasks  in  the  management  area  received  only  a  modest  investment 

Finally,  the  Panel  is  apparently  encountering  serious  difficulties  in  reconciling  different 
viewpoints  among  the  Military  Services  for  the  purpose  of  establishing  Joint  Service  standards.  For 
example,  the  need  for  an  improved  military  standard  for  TPS  development  and  acquisition  has  been 
recognized  since  1978.  Revisions  of  MIL-STD-1519  (USAF),  "Test  Requirements  Document."  have 
been  in  process  for  several  years,  but  the  revised  standard,  planned  for  mandatory  use  DoD-wide,  still 
has  not  been  approved  as  of  December  1985.  The  prospect  for  such  a  DoD-wide  standard  for  the  test 

4George  W.  Neumann,  "The  JLC  Panel  on  Automatic  Testing,”  1981  Proceedings  Annual 
Reliability  and  Maintainability  Symposium  (New  York:  The  Institute  of  Electrical  and  Electronics 
Engineers,  Inc  ,  1981 )  p  261 
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( 1982  Program  Review) 


Management 


NUMBER  OF  SUBTASKS- 


Acquisition  Support  Testing  Technology 


D 

c 

A 

I) 

FUNDING 
LEVEL i 
(FY81  -FY85) 


>$100M 
$10M  -  $100M 
$5M  -  $10M 
$1M  -  $5M 
<$1M 

Undetermined 


Total:  ~-$320M 


‘M  =  million.  FY  =  fiscal  year 

2C  =  completed,  A  =  active.  D  =  deleted 

requirements  document  appears  to  be  remote  because  of  irreconcilable  differences  of  opinion  among 
Military  Service  representatives  In  this  respect,  the  JLC  Panel  on  Automatic  Testing  has  not 
brought  the  DoD  closer  to  the  goal  of  a  uniform  standard  for  the  most  critical  document  in  the 
development  and  acquisition  of  TPSs. 

JLC  PANEL  ON  STANDARDIZATION  OF  AGSE 


The  JLC  Panel  on  Standardization  of  AGSE  was  chartered  in  March  1980  pursuant  to  the 
Comptroller  General’s  Report  to  the  Congress,  Increased  Standardization  Would  Reduce  Costs  of 
Ground  Support  Equipment  for  Military  Aircraft.  LCD-80-30,  7  February  1980,  and  related  task 
assignments  from  the  DoD  Ad  Hoc  Working  Group  for  Standardization  of  Aircraft  Ground  Support 
Equipment  (see  Chapter  3)  The  purpose  of  the  Panel  was  twofold:  ( 1 )  to  develop  joint  methods  to 
more  effectively  implement  current  policy  and  directives  addressing  acquisition  and  standardization 


of  AGSE  and  (2)  to  recommend  new  joint  directives  and  systems  where  existing  ones  inadequately 


promote  Joint  Service  standardization.5  In  accordance  with  its  study  plan,  which  was  approved  in 
December  1980,  the  Panel  established  three  subpanels  to  accomplish  specific  tasks 


•  Policies  and  Procedures  Subpanel:  Review  of  current  policies  and  procedures  for 
AGSE  standardization  and  development  of  recommendations  to  ensure  the  planning 
and  implementation  of  a  cost-effective  AGSE  standardization  program 

•  Data  Storage  and  Retrieval  Subpanel:  Review  of  existing  data  storage  and  retrieval 
systems  for  AGSE  with  emphasis  on  the  capability  of  those  systems  to  provide 
decisionmaking  information  to  contractors  and  Government  program  managers 
Development  of  recommendations  for  improvement  to  existing  systems  or  develop¬ 
ment  of  new  systems  to  increase  their  effectiveness  to  avoid  proliferation  of  AGSE 

•  Contract  Methods  and  Specifications  Subuanel  Examination  of  AGSE  contract 
methods  with  emphasis  on  defining  alternatives  for  maximizing  AGSE  standard¬ 
ization  within  existing  legal  and  regulatory  constraints  and  recommending 
revisions  to  legal  and  regulatory  restrictions  that  preclude  achievement  of  AGSE 
standardization.  Evaluation  of  the  feasibility  of  contract  incentives  to  encourage 
maximum  selection  of  common  AGSE  Evaluation  of  methods  for  follow-on 
procurement  of  AGSE  to  maximize  standardization  Review  of  AGSE  specifications 
and  standards  to  determine  their  adequacy  and  impact  on  achieving  standard¬ 
ization. 

The  Panel  completed  its  study  in  1983  with  the  publication  of  a  single  report,  which  integrated 
the  technical  reports  of  each  subpanel  . H 

Summary 

The  Panel  found  that  the  DoD  and  Military  Service  policies  and  directives 

are  in  place  to  adequately  promote  the  achievement  of  intraservice 
AGSE  standardization  (i  e  .  non  proliferation!  objectives  However,  the  full 
application  of  these  requirements  is  not  being  effected  due  to  constraints  on 
manpower  and  fiscal  resources  in  all  three  services  and  the  basic  organi¬ 
zational  alignments  in  the  Army  and  Air  Force  for  AGSE  management  7 


5GeneralJohn  R.  Guthrie.  Admiral  A.  J  Whittle,  -Jr  .  General  Brvee  Poe,  II  and 
General  Alton  D  Slav,  "Charter  for  Joint  DARCOM/NMC  AFI.C  A  ESC  Commanders  Panel  on 
Standardization  of  Aviation  Ground  Support  Equipment"  (Washington,  1)  C  Joint  Logistics 
Commanders,  19  March  1980) 

‘•’•Joint  DARCOM/NMC/AFLC/AFSC  Commanders  Panel  on  Standardization  of  Aviation 
Ground  Support  Equipment,  Final  Report  (Washington,  DC  -Joint  Logistics  Commanders, 
28  J  une  1 983) 

"Ibid,  SecMon  III,  "Summary  ”  Note  that  the  major  changes  m  Armv  TMDF  management,  as 
are  described  in  Chapter.1),  became  effective  in  1983,  after  completion  of  the  subpanel  technical 
reports  in  mid  1982 
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Beyond  the  need  for  increased,  dedicated  manpower  resources  (to  review  aircraft  specifications  and 
design  for  AGSE  compatibility,  identification,  and  selection  and  for  common  AGSE  development  and 
acquisition),  the  Panel  singled  out  two  factors  of  primary  importance  to  intraservice  AGSE  standard¬ 
ization:  ( 1 )  "the  existence  of  an  organizational  element  with  specific  authority,  responsibility  and 
accountability  for  the  achievement  of  stated  objectives”;  and  (2)  "the  faculty  for  developing  AGSE 
independently  from  weapon  system  project  auspices.”  With  respect  to  the  first  factor,  it  concluded 
that  both  the  Army  and  Air  Force  need  to  strengthen  their  management  structures  "to  allow  AGSE 
development/acquisition  decisions  to  be  made  based  on  total  service  aviation  support  factors  vice 
individual  weapon  system  support  factors  alone.”  With  respect  to  the  second  factor,  it  concluded  that 
both  the  Army  and  Air  Force  "require  the  establishment  of  research  and  development  budget  line 
items  specifically  designated  for  engineering  development  of  common  AGSE  ” 

While  the  same  factors  (manpower  resources,  fiscal  resources,  and  organizational 
alignment)  affect  interservice  AGSE  standardization,  the  Panel  found  it  is  'further  debilitated  by  the 
lack  of  specifically  defined  service  policy  on  responsibility  (except  for  multi-service  weapon  system 
projects)  for  interservice  coordination/execution  of  AGSE  development/acquisition  ”  The  Panel 
concluded  that  the  Military  Services  "must  establish  dedicated  interservice  coordination  positions, 
located  in  the  [samel  organizational  element  where  AGSE  selection  policies  and  decisions  are  made” 
and  that  SISMS  "must  be  revised  to  impose  I  its  I  application  on  single-service,  as  well  as  multiservice, 
weapon  system  projects  ” 

From  its  review  of  data  storage  and  retrieval  systems  for  AGSE,  the  Panel  found 
four  major  shortcomings:  (l)a  method  for  validating  contractor  compliance  with  Military  Service 
requirements  to  screen  existing  AGSE  data  bases  does  not  exist.  (2)  many  AGSE  data  bases  are  not 
available  to  contractors  or  other  Military  Services.  (3)  MIL  HDBK  (Military  Handbook)  300  data 
submission  requirements  cannot  be  enforced;  and  (4)  MIL, HDBK  300  needs  to  be  automated  and 
updated  so  it  can  be  more  readily  used 
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The  Panel  concluded  that 


.  .  the  services  need  to  revise  the  SISMS  procedures  to  include  provisions 
for  contractor  certification  of  data  base  screening,  and  improve  and  main¬ 
tain  the  structure,  application  and  use  of  MIL-HDBK-300  as  the  primary 
data  storage  and  retrieval  system  for  AGSE  identification  and  selection 
among  the  services. 

From  its  review  of  contract  methods  and  specifications  for  AGSE,  the  panel  found 
several  relationships  between  long-range  planning,  procurement  regulations,  contracting  methods, 
and  specification  currency  that  have  adversely  influenced  AGSE  standardization. 

The  Panel  found  that  the  Military  Services  seldom  use  the  Defense  Acquisition  Regula¬ 
tions  (DAR)  standardization  exception  clause  (DAR  3-213)  "due  to  the  extended  (2-6  months) 
processing  time  and  dubious  nature  of  the  outcomes  cf  requests  for  authority  to  negotiate  which 
currently  require  determinations  and  findings  (D&F)  by  the  Secretary  of  a  Department  ”  It  con¬ 
cluded  that  "Congressional  legislation  should  be  sought  to  allow  delegation  of  authority  below  the 
Secretarial  level,  to  the  head  of  the  procuring  activity  for  procurements  less  than  $50M  in  value”  and 
that  the  Military  Services  should  "encourage  cognizant  service  AGSE  acquisition  managers  to  make 
maximum  use  of  DAR  3-213  and  factor  the  administrative  lead  time  for  processing  the  (D&F)  into 
their  procurement  planning  ” 

The  Panel  also  found  that  multiyear  contracting  approaches  and  contractual  options  for 
additional  quantities  represent  the  primary  methods  used  by  the  Military  Services  to  ensure  stan 
dardization  but  that  "the  effective  use  of  these  contracting  methods  requires  precise  long  range 
planning  for  identification,  development  and  procurement  of  AGSE,  as  well  as  advanced  program 
ming  of  required  fiscal  resources  to  fully  support  contractual  commitments.”  It  concluded  that  the 
"services  need  to  emphasize  the  effective  conduct  of  this  long  range  planning  and  programming." 

The  Panel  observed  a  lack  of  experience  in  applying  incentive  clauses  to  weapons  system 
contracts  for  the  purpose  of  motivating  contractors  to  maximize  selection  of  common  AGSE  It 
concluded  that  "notwithstanding  the  additional  administrative  burden  associated  with  award  fee 
provisions,  the  potential  advantages  in  terms  of  AGSE  standardization  warrant  trial  application  to  a 


major  weapon  system  contract”  and  recommended  that  the  Joint  Service  Vertical  Lift  Aircraft  pro¬ 
gram  be  designated  for  this  purpose. 


Finally,  the  Panel  noted  that  AGSE  standardization  is  only  one  component  of  the  DSSP 

and  that  "the  existing  problems  with  the  upgrading  of  military  standardization  documents  are  not 

unique  to  AGSE,  but  are  generic  to  the  DSSP.”  It  concluded  that  AGSE  proliferation  would  not  be 

ameliorated  with  greater  emphasis  on  the  DSSP  effort  because  of  "the  incompatibility  of  the  DSSP,  as 

a  program  to  manage  standardization  of  DoD  equipment,  and  the  DAR.”  According  to  the  Panel,  the 

fundamental  problem  is  the  following: 

The  basic  DAR  philosophy  to  maximize  competition,  by  its  very  nature, 
results  in  proliferation.  Thus,  while  the  DSSP  gives  DoD  managers 
valuable  tools  to  control  the  inventory,  the  DAR  almost  precludes  their 
effective  use.  In  fact,  when  viewed  in  light  of  existing  DAR  philosophy,  the 
primary  function  of  specifications  becomes  promoting  competition  rather 
than  controlling  proliferation. 

The  Panel  concluded  that  the  JLC  should  request  a  "comprehensive  and  programmatic  review  of  the 
objectives  and  administration  of  the  DSSP ...  to  assess  their  relationships  to  the  achievement  of 
material  standardization.” 

The  Panel’s  recommendations,  which  are  summarized  in  Figure  4-2,  were  approved  by 
the  JLC.  They  also  agreed  to  implement  10  of  the  12  recommendations  "within  each  of  our  Com 
mands  to  the  maximum  reasonable  extent  as  resources  permit”  and  to  refer  action  on  two  recommen¬ 
dations  to  the  Under  Secretary  of  Defense  for  Research  and  Engineering  Even  though  the  Panel 
focused  on  AGSE,  we  believe  that  its  recommendations  are  applicable  to  all  support  equipment, 
including  test  equipment. 

Follow-Up 

The  Panel  was  also  tasked  to  monitor  the  implementation  of  its  recommendations  and 
report  progress  to  the  JLC.  We  have  no  specific  information  on  the  current  status  of  implementation. 
However,  the  House  Committee  on  Government  Operations,  in  a  report  previously  referenced  in 
Chapter  3,  concluded  that  implementation  had  not  begun  as  of  January  1984  More  recently,  the  JLC 
awarded  contracts  to  automate  and  integrate  AGSE  data  bases,  including  MIL-HDBK-300:  the 
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FIGURE  4-2.  SUMMARY  RECOMMENDATIONS  OF  JLC  PANEL 
ON  STANDARDIZATION  OF  AGSE 


1.  Establish  dedicated  inter-Service  coordination  positions  in  each  Military  Service,  located  in  the  organizations 
where  AGSE  selection  policies/decisions  are  made,  and  having  direct  responsibility  for  interservice  AGSE 
standardization. 

*2.  Seek  Congressional  legislation  to  delegate  authority,  below  the  secretarial  level,  to  make  determinations  and 
findings  with  respect  to  entering  into  contracts  by  negotiation  under  DAR3-213  and  revise  DAR  Section  III. 
Part  3,  paragraphs  3-302  and  3-303  accordingly 

3.  Revise  procedures  for  contractor  review  of  AGSE  dat_  bases  and  submission  of  AGSE  recommendations  to  the 
Military  Services,  as  established  by  SISMS,  as  follows. 

•  Require  contractors  to:  (a)submit  AGSE  candidate  lists,  iblscreen  Military  Service  Standard  Item 
lists/PILs.  and  (c)  certify  screening  accomplished  on  each  SERD; 

•  Prepare  new/revise  existing  Data  Item  Descriptions;  and 

•  Enforce  SISMS. 

4.  Encourage  cognizant  Military  Service  AGSE  acquisition  managers  to:  (l)make  standardization 
consideration  a  major  technical  factor  in  source  selection  criteria,  (2)  make  effective  use  of  both  DAR  3  213 
and  multiyear  or  option  contracts  for  procurement  of  AGSE;  (3)  utilize  life  cycle  cost  analysis  in  AGSE 
selection  to  substantiate  any  deviations  from  standardization  policy;  and  (4)  emphasize  long-range  planning 
to  identify,  develop,  and  standardize  AGSE . 

5.  Designate  the  Joint  Service  Vertical  Lift  program  as  a  prototype  for  evaluation  of  an  award  fee  concept  to 
achieve  AGSE  standardization. 

6.  Strengthen  AGSE  organizational  elements  within  Army  and  Air  Force  to  focus  authority  and  responsibility 
for  achieving  AGSE  standardization. 

7.  Establish/increase  dedicated  manpower  resources  within  each  Military  Service  for  aircraft  specification  and 
design  review  for  AGSE  compatibility,  AGSE  selection,  and  management  of  common  support  equipment 
acquisition. 

8.  Establish  a  separate  Program  Element  in  Air  Force  and  Army  budgets  dedicated  to  engineering  development 
of  common  AGSE. 

9.  Improve  and  maintain  the  structure,  application,  and  use  of  MIL  HDBK  300  as  the  primary  data  storage  and 
retrieval  system  for  AGSE  identification  and  selection  among  the  Military  Services. 

*10.  Recommend  OSD  conduct  a  pragmatic  review  of  the  objectives  and  administration  of  the  DSSP  and  its 
relationship  to  the  achievement  of  AGSE  standardization 

11.  Encourage  aircraft  and  airborne  system  acquisition  managers  to  consider  AGSE  proliferation  impact  in 
selecting  system  design  alternatives  and  to  impose  design  specification  controls  on  aircraft  to  AGSE 
interfaces  to  allow  interoperability  of  various  AGSE  on  different  aircraft. 

12.  Impose  SISMS  requirements  in  all  acquisition  and  harmonize  associated  Data  Item  Descriptions  to  ensure 
conformity. 

*  Action  referred  to  the  Undersecretary  of  Defense  for  Research  and  Engineering. 
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long-term  goal,  reportedly,  is  the  development  of  a  comprehensive  DoD-wide  data  base  management 
system  on  nonconsumable  items  in  the  DoD  inventory. 

JOINT  TECHNICAL  COORDINATING  GROUP  FOR  METROLOGY  AND  CALIBRATION 

The  JTCG-METCAL  was  established  in  1967  to  coordinate  the  metrology  and  calibration 
programs  within  the  DoD.  That  group  was  tasked  to  promote  standardization  and  uniformity  among 
the  Military  Services.  Since  the  establishment  of  a  separate  Subgroup  for  Consolidation  of  Cali¬ 
bration  Services  in  June  1975,  the  JTCG-METCAL  has  conducted  a  series  of  studies  of  the  feasibility 
and  cost  effectiveness  of  consolidation  on  a  regional  basis.  However,  many  of  its  recommendations 
have  not  been  implemented.  For  example,  its  recommendations  for  the  restructuring  of  calibration 
services  in  Europe  have  not  been  approved. 8  The  GAO  has  routinely  urged  the  OSD  to  centralize 
management  and  consolidate  calibration  and  diagnostic  programs  (e  g  ,  oil  analysis).9  Recommen¬ 
dations  to  replace  the  individual  centers  of  expertise  in  metrology  and  calibration  within  the  Military 
Services  by  a  single  Metrology  and  Calibration  Center  assigned  to  the  Joint  Chiefs  of  Staff  date  back 
to  1973. 10  The  proposed  center  would  be  responsible  for  (1)  developing  and  managing  an  integrated 
metrology  and  calibration  program,  (2)  selecting  all  standards  and  calibration  equipment,  (3)  pro¬ 
curing  and  distributing  all  standards,  (4)  conducting  research  and  development,  (5)  developing  and 
promulgating  standard  calibration  procedures,  (6)  establishing/reviewing  calibration  intervals  for  all 
ETE  and  calibration  standards,  (7)  auditing  and  evaluating  calibration  facilities,  (8)  determining 
location  and  ownership  of  all  calibration  laboratories/facilities/teams,  (9)  determining  technical 


8Joint  Technical  Coordinating  Group  for  Metrology  and  Calibration,  Subgroup  for  Consolida¬ 
tion  of  Calibration  Services,  Consolidation  of  Department  of  Defense  Calibration  Facilities  in 
Europe  (Redstone  Arsenal,  Alabama:  U.S.  Army  Metrology  and  Calibration  Center,  September 


1976) 


9U.S.  General  Accounting  Office,  A  Central  Manager  Is  Needed  To  Coordinate  The  Military 
Diagnostic  and  Calibration  Program.  LCD-77-427  (May  1977);  and  U.S.  General  Accounting  Office, 
Centralized  Direction  Needed  For  Calibration  Program,  LCD-77-426  (Washington,  D  C..  U  S 
General  Accounting  Office,  June  1977) 


10Major  James  E.  Deal  (U.S.  Air  Force),  "An  Examination  of  the  Metrology  and  Calibration 
Programs  of  the  Three  Military  Departments,”  Research  Study  (Maxwell  Air  Force  Base,  Alabama: 
Air  University,  May  1973). 
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training  requirements,  and  (10)  representing  the  DoD  single  point-of-contact  with  the  National 
Bureau  of  Standards. 


Charter  and  Organization 


The  current  JTCG-METCAL  charter  states  its  purpose  as  follows:1 1 

I.  PURPOSE.  Provide  emphasis  on  improvements,  interservice  coordi¬ 
nation  and  cost  reduction/avoidance  of  metrology  and  calibration  opera¬ 
tions  throughout  the  services.  Ensure  accomplishment  of  necessary 
interservice  actions  and  coordination  on  specifications/standards,  direc¬ 
tives,  equipment  development  and  acquisition,  data  systems,  training, 
procedures/documentation,  interval  determination,  engineering,  inter¬ 
service  support,  and  other  issues  relating  to  metrology  and  calibration. 

Determine  need  for  present  and  additional  DOD  calibration  facilities  and 
resources,  and  maintain  liaison  with  the  National  Bureau  of  Standards  on 
requirements  for  calibration  and  calibration  engineering  services. 

The  JTCG-METCAL  consists  of  a  command  representative  and  alternate  from  each  of 
the  four  participating  commands,  with  the  Marine  Corps  and  the  Defense  Logistics  Agency  repre¬ 
sented  as  invited  participants. 

Recent  Studies 

A  study  plan  for  the  JTCG-METCAL,  approved  by  the  Joint  Secretariat  in  March  1983, 
identified  the  following  tasks:  12 

A  Task  A:  Prepare  a  Joint  Service  Regulation  ( JSR)  to  implement  that 
portion  of  DOD  Directive  4155  1  relating  to  interservice  coordination  of  the 
DOD  metrology  and  calibration  programs 

1  Revise  the  Handbook  of  Information  and  Guidelines  (HIG) 
which  provides  the  CRTs  [Calibration  Review  Teamsl  with  guidelines, 
methodology  and  criteria  for  conducting  calibration  reviews. 

2.  Develop  a  proposed  Joint  Service  Regulation  (JSR)  which 
prescribes  policies  and  procedures  for  interservice  coordination  of  the  DOD 


1  •General  John  R.  Guthrie;  Admiral  J.  G  Williams,  Jr.  General  Bryce  Poe,  II  and 
General  Robert T  Marsh,  "Charter  for  Joint  DARCOM/NMC/AFLC/AFSC  Commanders'  Joint 
Technical  Coordinating  Group  on  Metrology  and  Calibration  (JTCG-METCAL)”  (Washington,  DC 
Joint  Logistics  Commanders,  29  July  1981 ) 

12Fred  B  Seeley  (DARCOM  (U  S  Army  Development  and  Readiness  Command)  Memberl, 
Joseph  T  Siedlecki  (Naval  Material  Command  Member  and  Chairman),  Selden  W  McKnight 
(AFLC  Member),  and  Major  M  J  Murtaugh  (AFSC  Member),  "Study  Plan  Joint  Technical 
Coordinating  Group  on  Metrology  and  Calibration  (JTCG-METCAL),’’  Unpublished  Working 
Paper,  22  November  1982. 


metrology  and  calibration  programs;  coordinate  the  JSR  and  submit  for 
publication. 

B.  Task  B:  Establish  policies  and  procedures  to  enhance  the  use  of 
common,  interchangeable  calibration  procedures. 

1.  Obtain  interservice  approval  of  calibration  requirements 
document  (CRD)  format/contents,  consistent  with  maintenance  policies  of 
individual  services;  complete  initial  CRDs  for  interservice  use. 

2.  Develop  appropriate  DOD  specification/documentation  for 
CRD  preparation. 

3.  Analyze  interservice  applications  for  other  generic  documents, 
i.e.,  measurement  method  module  and  test  instrument  procedures. 

4.  Prepare  documents  for  selected  measurement  systems/test  in¬ 
struments  for  interservice  use. 

5.  Establish  policies  and  procedures  to  ensure  common,  inter¬ 
changeable  calibration  procedures  are  used  wherever  possible. 

C.  Task  C:  Review  current  methods  for  determining  calibration  inter¬ 
vals  used  by  each  service  with  the  viewpoint  of  standardizing  methodology  , 
investigate  the  feasibility  of  standardizing  intervals  on  like  equipment  in 
the  DOD  inventory  and  policy  for  common  reliability  targets. 

1.  Select  sample  of  high  workload  test  equipment  from  the 
three  services  and  compare  calibration  intervals;  identify  and  analyze 
differences  to  determine  if  they  are  the  result  of  procedures,  usage, 
environment  or  interval  policy;  report  finding  and  recommendations. 

2.  Investigate  feasibility  and  impact  of  DOD  standard  policy  on 
calibration  intervals. 

D  Task  D:  Establish  a  coordinated  DOD  measurement  technology  and 
standards  research,  development  and  engineering  program  and  update 
annually;  coordinate  funding  and  evaluate  performance;  establish  National 
Bureau  of  Standards  (NBS)  projects,  as  required. 

1  Establish  and  update  annually  the  DOD  measurement  tech 
nology  and  standards  research,  development  and  engineering  program 

2.  Establish  engineering  projects  at  NBS  and  with  other  indus¬ 
trial  and  professional  groups  as  appropriate;  evaluate  performance. 

E.  Task  E.  Establish  uniform  procedures  for  determining  DOD  require¬ 
ments  for  calibration  services  to  be  provided  by  the  National  Bureau  of 
Standards  (NBS);  estimate  cost,  coordinate  budgeting/funding  and  estab¬ 
lish  a  schedule  for  NBS  services 

1.  Develop  schedule  for  NBS  calibration  services. 

2.  Coordinate  funds  for  NBS  calibration  services. 

F  Task  F  Complete  reviews  of  possible  duplication  of  calibration 
services  and  make  recommendations  for  consolidation  of  DOD  calibration 
facilities  or  increased  use  of  interservice  support  agreements  where 
appropriate 


Initiate  calibration  review  studies  for  those  areas  listed  below;  com¬ 
plete  the  study,  submit  draft  study  for  OPR  [office  of  primary  responsi¬ 
bility]  review  and  prepare  the  final  study;  coordinate  the  study  with  major 
commands  and  submit  the  final  study  and  proposed  agreement  to  JLC  for 
approval. 

1.  Philadelphia 

2.  Washington  DC 

3.  North  Central  (North  Dakota,  South  Dakota,  Northern 

Nebraska,  Minnesota,  Wisconsin,  Northern  Iowa) 

4.  Southwest 

5.  Carolinas 

6.  New  England 

7.  Southeast  Georgia/Florida 

8.  West  Central 

9  North  Central  (Michigan,  Indiana,  Ohio,  Northern  Kentucky, 

Western  West  Virginia,  Western  Pennsylvania) 

10.  Central  CONUS  [Continental  United  States! 

11.  South  Central 

G.  TaskG.  Perform  a  feasibility  study  to  determine  potemtial  [sic]  uses 
and  cost  benefits  of  a  common  data  base  for  the  services;  determine  poten¬ 
tial  costs  of  developing  and  maintaining  such  a  data  base. 

1  Perform  feasibility  study  to  determine  potential  uses  and  cost 
benefits  of  a  common  data  base  for  the  Services;  determine  potential  costs  of 
developing  and  maintaining  a  common  data  base. 

2  Submit  feasibility  study  to  OPRs  for  review  and  comment 
Status  and  Comments 

We  do  not  have  any  specific  information  on  the  status  of  the  above  tasks  other  than 
Tasks  C  and  D  With  respect  to  Task  C,  the  Calibration  Coordination  Group  found  dramatic 
differences  among  the  Military  Services  in  calibration  intervals  for  like  equipment.  The  factors 
contributing  to  those  differences  are  summarized  in  Figure  4-3.  Until  those  differences  are  resolved, 
however,  a  DoD  standard  policy  on  calibration  intervals  will  not  be  feasible.  Furthermore,  the 
differences  also  tend  to  preclude  consolidation  of  the  metrology  centers,  as  urged  by  the  GAO  With 
respect  to  Task  D,  the  Calibration  Coordination  Group  prepared  the  first  edition  of  the  Tri-Service 
Metrology  Research.  Development,  and  Engineering  Plan  in  1983,  specifying  measurement  needs  in 
electro-optics,  microwave/millimeter  wave,  electronics/mechanical/physical  test  equipment,  test 


FIGURE  4-3.  SERVICE  DIFFERENCES  IN  CALIBRATION  INTERVAL  DETERMINATION 


1.  End  of  Period  (EOP)  versus  Average  Over  Period  (AOP)  Reliability: 

a.  Air  Force  EOP  reliability  is  85  percent  with  an  AOP  reliability  of  approximately 
95  percent. 

b.  Navy  EOP  reliability  is  approximately  73  percent  with  an  AOP  reliability  of 
85  percent. 

c.  Army  EOP  is  approximately  75%  reliability  with  an  AOP  of  85%  reliability 

d.  Calibration  interval  difference  for  EOP  versus  AOP  is  a  factor  of  two. 

2.  Procedures  for  same  part  number  cems  show  significant  difference  (preliminary  study 
results). 

3.  Army  and  Air  Force  do  not  have  a  dog-and-gem  removal  process  before  running  of  their 
algorithms.  Navy  claims  a  20%  extension  with  this  technique 

4.  Army  and  Navy  depots  do  not  consider  repair  data  in  their  analysis  only  in  tolerance 
and  out-of-tolerance  condition  Their  repairs  often  take  place  at  different  locations  than  those 
at  which  the  calibrations  are  performed 

5  Key  data  elements  in  Navy  and  Army  programs  are  part  number  and  manufacture 
code.  Air  Force  uses  part  number  and  work  unit  code 

6.  Army  and  Air  Force  accumulate  time  using  a  renewable-at  failure  concept,  where  as 
Navy  uses  a  renewable-at-calibration  concept 

7  Exponential  Distribution  Statistics  are  now  being  used  by  all  three  Military  Depart¬ 
ments  (recently  standardized).  Previously  Air  Force  and  Army  used  Weibull  Distribution 
Statistics. 

8  Navy  produces  a  dog-and-gem  list  with  a  suggested  different  interval  than  the  part 
number  family;  Air  Force  and  Army  do  not 

9  Army  adjusts  intervals  to  nearest  120  days  to  correspond  to  calibration  van  trips  Air 
Force  and  Navy  adjust  to  nearest  month 

10.  Air  Force  publishes  TO  [Technical  Orderl  33K- 1-100,  calibration  interval  changes, 
twice  a  year;  Army  publishes  once  a  year;  and  Navy  makes  monthly  adjustments 

1 1 .  Methods  for  establishing  initial  calibration  intervals  are  different. 

12.  Review  criteria  for  suggested  interval  changes  are  different 


SOURCE:  Calibration  Interval  Working  Group  of  the  Calibration  Coordination  Group,  'Areas 
of  Significant  Difference”  (undated  working  paper). 


5.  MILITARY  SERVICE  INITIATIVES 


This  chapter  reviews  the  major  studies  and  initiatives  undertaken  by  the  Military  Services  in 
recent  years  to  improve  the  management  and  support  of  test  equipment. 

ARMY  INITIATIVES 

In  1979,  the  Army  introduced  an  improved,  more  effective  TMDE  calibration  and  repair 
concept.  Previously,  TMDE  calibration  and  repair  had  been  characterized  by  (1)  split  responsibility 
for  calibration  and  repair  of  TMDE,  and  (2)  complicated  support  channels  arising  from  two  levels  of 
calibration  in  the  field  (below  the  secondary  reference  standards).  The  new  concept  provided  total 
TMDE  calibration  and  repair  support  from  a  single  source  on  an  area  basis,  with  AMC  management 
and  control  of  all  calibration  and  repair  of  common  TMDE  and  selected  special  TMDE.1  The 
transition  began  with  U  S.  Army  Europe  (USAREUR)  in  October  1979,  followed  by  Korea  and  the 
United  States. 

A  second  initiative  was  AMC’s  decision,  formally  announced  in  December  1979,  to  standardize 
Army  ATE  by  adopting  the  AN/USM-410  EQUATE  (Electronics  Quality  Assurance  Test  Equipment) 
as  the  Army’s  standard  general-purpose  ATE  for  use  at  general-support  and  depot  levels  The  rules 
for  determining  whether  system-peculiar  ATE  already  planned  or  in  development  could  deviate  from 
the  standard  ATE  policy  were  promulgated  by  AMC  in  1980  2  Those  rules,  in  effect,  emphasized  the 
benefits  of  standardization  and  approved  a  "one-shot”  cost  increase,  if  necessary,  to  convert  planned 
ATE  to  AN/USM  410  when  long-term  cost  or  readiness  benefits  could  be  anticipated 

A  third  initiative  was  the  TMDE  Modernization  Program  developed  jointly  by  the  U  S  Army 
Communications-Electronics  Command  (CECOM)  and  the  Army  Signal  Center  in  1980  to  "correct 


’Department  of  the  Army,  DA  Concept  Study  for  Improved  Army-wide  TMDE  Calibration 
and  Repair  Operations  (Washington.  D  C  Department  of  the  Army,  March  1977) 


the  technical  inadequacy,  obsolescence  and  proliferation  within  the  present  inventory.”3  Under  this 
program,  obsolete  manual  ETE  of  many  different  makes  and  models  was  replaced  by  a  much  smaller 
variety  of  modern  commercial  test  equipment.  The  program  was  originally  scheduled  to  be  executed 
over  a  5-year  period  at  a  cost  of  $184  million  (CECOM-managed  TMDE  items  only),  but  manpower 
and  funding  constraints  caused  it  to  be  extended  over  a  10-year  period.  In  the  program's  first  year 
(fiscal  year  1981),  with  a  budget  of  $22  million  (out  of  $38  million  requested),  5  items  (types  of  test 
equipment)  were  procured  (total  count  of  3,600),  replacing  21  different  makes/models  in  the 
inventory.  In  fiscal  year  1982,  10  items  were  procured,  replacing  650  makes/mo*’els.  With  the  TMDE 
management  reorganization  in  1982,  program  responsibility  was  transferred  to  the  product  manager, 
TMDE  Modernization  (TEMOD),  and  the  program  expanded  to  non-CECOM  items 

A  fourth  initiative  was  the  establishment  of  a  TMDE  centralized  management  structure  by 
Secretary  of  the  Army  Charter,  April  1982.  That  Charter  designated  the  Commanding  General, 
AMC,  as  the  Department  of  the  Army  TMDE  Executive  Agent  responsible  for  total  Army  TMDE 
acquisition,  logistics,  and  financial  management  in  accordance  with  Army  Regulation  (AR)750  43 
("Test,  Measurement,  and  Diagnostic  Equipment”),  and  the  Deputy  Commanding  General  for  Mate¬ 
riel  Readiness  as  the  Executive  Director  for  TMDE  with  full  line  authority  (AR750  43  defines 
TMDE  in  the  broadest  sense  to  include  system  BIT/BITE,  ATE,  as  well  as  manual  ETE  )  The  result 
ing  management  structure  is  illustrated  in  Figure  5-1,  and  includes  the  following  three  key  players 

•  Centralized  TMDE  Activity  (CTA),  Lexington,  Kentucky 

•  U  S  Army  TMDE  Support  Group,  Huntsville,  Alabama 

•  Program  Manager  for  TMDE  ( PM,  TMDE),  Fort  Monmouth,  New  .Jersey 

The  responsibilities  of  each  are  indicated  in  Figure  5- 1  This  centralization  of  the  A  rmv’s  tradition¬ 
ally  dispersed  TMDE  management  structure  had  been  recommended  by  numerous  previous  studies 

3U  S  Army  Signal  Center,  "Test,  Measurement,  and  Diagnostic  Equipment  Modernization 
Fact  Sheet”  (Fort  Gordon,  Georgia  U  S  Army  Signal  Center,  May  1981) 


FIGURE  5-1  TMDE  MANAGEMENT  CONCEPT 
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including  one  as  recent  as  1979, 4  and  was  reiterated  by  a  more  recent  study  conducted  by  the 
Department  of  the  Army  TMDE  Action  Team  (DAT AT),  directed  by  the  Deputy  Chief  of  Staff  for 
Logistics  at  the  request  of  the  Assistant  Secretary  of  the  Army  (Installations,  Logistics,  and  Financial 
Management)  5  The  DATAT  report,  besides  recommending  a  centralized  management  structure, 
made  numerous  other  recommendations  for  improving  the  Army’s  management  and  support  of  test 
equipment 

The  Army's  fifth  initiative  was  the  formal  approval  of  the  DATAT  report  recommendations  and 
institution  of  a  plan  to  monitor  Army-wide  implementation.  Table  5-1  summarizes  the  initiatives 
that  have  been  or  are  being  taken  as  a  result  of  the  DATAT  effort.  From  the  numerous  initiatives, 
identified  in  Table  5-1,  several  are  singled  out  for  further  comment. 

•  Policy:  The  new  AR  750-43,  April  1984,  provides  improved  procedures  for  requirements 
identification,  selection,  acquisition,  and  life  cycle  support  for  all  types  of  TMDE 

•  Systems  Approach:  Traditionally,  TMDE  has  been  acquired  and  fielded  on  an  incremental 
basis.  The  TEMOD  program  is  organized  similarly,  replacing  obsolete  items  with  func¬ 
tionally  similar  ones  on  a  one-for-one  basis  The  systems  approach,  in  contrast,  consists  of 
determining  the  mission  requirements  of  a  maintenance  organization  and  the  best  mix  of 
TMDE  to  meet  those  requirements.  A  reduction  in  overall  numbers  ofTMDE  is  anticipated 
from  this  approach  The  PM,  TMDE  is  pursuing  this  approach,  starting  in  fiscal  year  1985 
with  the  High  Technology  Motorized  Division 

•  TEMOD  Program  The  TEMOD  program  was  originally  focused  on  FSC  6625  for  which 
CECOM  is  the  inventory  control  point.  In  September  1983,  a  master  plan  was  completed  to 
extend  TEMOD  to  other  FSCs,  but  TEMOD  programmed  funding  is  still  limited  to  Phase  I 
(FSC  6625  2,543  line  items) 

•  TMDE  Support  Evaluation  of  TMDE  support,  including  automatic  calibration,  was 
recently  completed  in  a  comprehensive  study  mandated  by  Congress. *> 

•  Standard  ATE  Policy  The  Army's  Direct  Support  Automated  Test  Support  Systems  pro¬ 
gram,  renamed  the  intermediate  forward  test  equipment  (IFTE)  program,  has  suffered 
several  delays  Fiscal  year  1985  funding  for  IFTE,  the  Simplified  Test  Equipment- 
Extended  iSTK  X'.  and  the  Army  Test  Technology  Laboratory  was  first  eliminated  by  the 


4Lieutenant  Oeneral  -Joseph  M  Heisor  lr  i  Ket  ),  Assessment  of  DA/DARCOM  Test.  Meas¬ 
urement.  and  Diagnostic  Euintnnent  Program  Washington,  I)  C  Department  of  the  Army, 
September  1 9791 


‘’Department  ol  the  Army  fesi ,  Measurement  and  Diagnostic  Equipment  Action  Team 
(DATAT)  Final  Report '  Washington,  I)  ('  Department  of  the  Army.  March  1982). 

‘'Michael  C  Sandusky  'Deputy  Executive  Director  for  TMDE),  Final  Comprehensive  Report 
on  the  L  S  Army  Calibration  Program  '  Washington,  I)  C  Headquarters  AMC,  July  1984) 


TABLE  5-1.  SUMMARY  OF  ARMY  TMDE  MANAGEMENT  INITIATIVES 


CATEGORY 

DESCRIPTION 

Management 

Structure 

Establishment  of  a  centralized  TMDE  management  structure  within  AMC  responsible  for  all 
TMDE  planning,  programming,  and  budgeting.  Establishment  of  the  Executive  Director  for 
TMDE  as  the  sole  proponent  for  all  TMDE  budget  actions. 

Management 

Information 

Systems 

Determination  of  management  information  needs.  Improvements  of  existing  PIL  (updating, 
elimination  of  obsolete  items,  and  automation)  and  TMDE  register.  Development  of  Armv-wide 
TMDE  asset  profile.  Development  of  central  Test  Equipment  Management  Information  System 
with  implementation  planned  for  mtd-1985. 

Development  of  automatic  management  information  system  network  interrommand  and  intra- 
command  transfer  and  retrieval  ofTMDE  data. 

Identification  of  TMDE  related  readiness  problems  by:  designating  selected  TMDE  as  "pacing 
items"  or  equipment  readiness  code  "A"  items  for  unit  status  reporting  purposes;  alerting  units  to 
emphasize  TMDE  in  readiness  assessments;  and  including  TMDE  in  Sample  Data  Collection 
studies. 

Policy 

and 

Procedures 

Revision  of  policy  to  strengthen  procedures  for  requirements  identification,  review  and  approval, 
acquisition,  and  support  for  all  TMDE,  including  special-purpose  TMDE  Development  and  insti 
tution  of  a  "systems  approach"  for  determining  TMDE  requirements  for  maintenance  organiza¬ 
tions. 

Minimization  ofTMDE  proliferation  by  authorizing  sole-source  reprocurement  and/or  multiyear 
procurements  of  preferred  TMDE  (PIl.i  in  accordance  with  the  escape  clause  in  DAR  3-213  ("intent 
to  standardize"). 

Development  of  source  selection  criteria  for  TMDE  procurement  including  life  cycle  costing  and 
adverse  impacts  of  prop,  letary  rights 

Implementation  and  enforcement  ofTMDE  issue  and  turn-in  procedures. 

Revision  of  policy  to  require  separate  type  classification  of  all  nonexpandable  TMDE  used  by 
military  units,  including  TMDE  components  of  "sets.  kits,  and  outfits."  and  to  monitor  their  turn-in 
for  calibration. 

Development  of  a  TMDE  Review  Handbook  to  formalize  review  procedures/methods  for  field 
visits/surveys.  Improvements  of  feedback  procedures,  including  reports  on  TMDE  Activities  and 
Lessons  Learned.  Revision  of  policy  to  improve  up-front  analysis  of  support  requirements  and  to 
require  TMDE  Support  Group's  certificate  of  supportability  prior  to  release  ofTMDE  in  order  to 
ensure  that  fielded  TMDE  can  be  supported.  Development  of  provisioning  procedures  for 
commercial  TMDE  to  ensure  initial  fielding  is  supported  with  spares  and  repair  parts. 

Modernization 

Expansion  and  increased  funding  support  of  the  TMDE  modernization  program 

Support 

Evaluation  ofTMDE  support,  including  automated  calibration,  to  determine  optimum  TMDE 
support  structure  for  Active  Army  and  Reserve  Components.  Implementation  of  needed  changes  in 
Army  National  Guard  and  Reserve  support  concepts  under  mobilization  condit  ions.  < 'ontinuation 
and  completion  of  consolidation  studies  under  auspices  of  the  JTCG-METCAC 

Automatic 

Testing 

Development  of  a  TPS  cost  estimating  approach  and  assessment  of  in  house  TPS  development 
c  apabilities.  Assessment  of  ATLAS  compiler  for  AN/CSM-410  and  IFTE  Establishment  of  stan¬ 
dard  ATE  policv.  including  MSM-105  (i.e.,  van  mounted  A.VL’SM  4 1 0 >  at  intermediate  'tear'. 
IFTE  at  intermediate  ^forward),  and  STE-X  at  organizational  level  m  support  ot  combat  and 
tracked  vehicles  Publication  ofan  Operational  and  Organizational  Plan  for  ATE  employment. 

Evaluation  of  cost-effectiveness  and  readiness  impact  of  field-level  repairs  of  printed  circuit  boards. 

Participation  in  Air  Force  and  Navy  ATE  programs  through  the  Army  TMDE  Technology  Team. 

Establishment  ofan  automatic  testing  technology  research  and  development  program  that  will 
meet  the  Army's  needs  in  the  following  areas:  design  for  testability,  programmable  interface 
electronics,  advanced  BIT/BITE  concepts,  nonelectronic  equipment  measurement  and  testing, 
improved  methodologies  for  digital  testing  programs,  and  resolution  of  test  problems  associated 
with  electro-optics,  microwave/millimeter  wave  equipment, and  solid  state  microelectronics 

SOURCE:  Test.  Measurement  and  Diagnostic  Equipment  iTMDE)  Implementation  Plan.  Department  of  the 
Army,  July  1982  This  plan  provides  the  implementation  schedule  and  milestones  for  recommendations  of  the  Department 
of  the  Army  TMDE  Action  Team  < DATAT)  Final  Report.  March  1982,  as  approved  by  the  Secretary  of  the  Army.  April 


Senate  Armed  Services  Committee,  later  approved;  IFTE  funding  was  next  withheld  by  the 
Army  Secretariat,  pending  further  justification  of  need,  and  finally  released  in 
January  1985.  The  IFTE  program  entered  engineering  development  with  contract  award 
to  Grumman  Aerospace  in  September  1985. 

A  Final  initiative,  with  a  much  broader  impact  than  just  TMDE,  was  the  Army’s  change  in 
maintenance  doctrine  in  1982.  Specifically,  the  elimination  of  general  support  maintenance  units 
from  the  corps  area,  as  recommended  by  previous  studies,7  and  their  replacement  by  intermediate 
(rear)  units  behind  the  corps  rear  boundary  (i.e. ,  in  echelons  above  corps)  has  a  far-reaching  impact  on 
ATE  requirements.  With  less  mobility  required  at  echelons  above  corps,  the  deployment,  operational 
use,  and  support  of  the  AN/MSM-105  becomes  more  viable.  At  the  same  time,  the  doctrine  mandates 
highly  mobile  ATE  for  assembly  testing  and  repair  by  intermediate  (forward)  maintenance  units 
(formerly  direct  support)  in  corps  (nondivisional  units),  division,  and  forward  areas  In  this  context,  it 
is  noteworthy  that  L'SAREUR’s  policy  is  to  screen  printed  circuit  boards  (PCBs)  if  possible,  evacuate 
PCBs  that  cannot  be  screened  or  screened  as  "no-go”  to  the  21st  Support  Command,  and  repair  them 
at  the  Pirmasens  Communications-Electronics  Maintenance  Center  8 
N  AVY  INITIATIVES 

The  Navy’s  initiatives  to  improve  management  and  support  of  test  equipment  can  be  summa¬ 
rized  in  two  main  themes.  For  ETE,  the  focus  has  been  on  ensuring  better  coordination  among  the 
Systems  Commands;  for  ATE,  the  focus  has  been  on  advancing  the  testing  technology  used  in  the 
Fleet  and  developing  the  associated  management  support  tools 

Manual  Test  Equipment 

Navy  initiatives  to  control  or  reduce  unnecessary  proliferation  of  general-purpose  ETE 
(GPETE)  date  back  to  1969  when  N'AVMAT  delegated  centralized  GPETE  management 


7General  John  R  Guthrie  (Study  Director),  Army  Logistics  1981  Study  (Washington,  D  C 
Department  of  the  Army,  August  1981 ) 

8Commander  in  Chief  U  S  Army,  Europe  Message  DTG  231059Z  Subject  "Theater  Repair 
Policy  for  Printed  Circuit  Boards  (PCBs),”  February  1984.  The  policy  is  articulated  in  more  detail 
in  USARECR  Supplement  1  to  AR  750  1 


responsibility  to  the  Naval  Electronic  Systems  Command  (NAVELEX).9  Within  NAVELEX,  that 
responsibility  was  centralized  in  the  TMDE  Division  (ELEX-841),  including  "research,  design, 
development,  evaluation,  testing,  logistics  planning,  acquisition,  inventory  management,  mainte¬ 
nance,  repair,  calibration,  budgeting,  funding,  requirements  determination,  allocation,  allowance, 
delivery,  and  follow-on  support.”10  In  exercising  its  authority,  NAVELEX  was  unable  to  obtain 
agreement  on  what  constitutes  GPETE.  As  a  result,  Systems  Commands,  program  management 
offices,  field  activities,  and  the  Fleet  continued  with  procurement  of  ETE  under  the  guise  of  special- 
purpose  ETE  (SPETE),  beyond  NAVELEX  control.  NAVMAT  solved  this  problem  in  1973  by  estab¬ 
lishing  an  ETE  Board,  responsible  for  determining  the  classification  ( i.e. ,  either  GPETE  or  SPETE)  of 
ETE  in  the  inventory  as  well  as  new  requirements  whenever  agreement  could  not  be  reached  by  the 
cognizant  Systems  Command  and  NAVELEX.11  Eventually,  NAVMAT  established  a  standard  item 
list  [MIL-STD-1364  (Navy),  "Standard  General  Purpose  Electronic  Test  Equipment”),  with 
NAVELEX  responsible  for  keeping  the  list  up  to  date,  and  restricting,  to  the  extent  practical,  new 
GPETE  procurements  to  items  listed  in  MIL-STD-1364.  Specific  procedures  are  prescribed  for  pro¬ 
curement  of  "non-standard  GPETE”  (i.e.,  items  classified  as  GPETE,  but  not  listed  in  the  latest 
version  of  MIL-STD-1364). 12 


9National  Material  Command,  "Naval  Material  Command  Organization  Manual,” 
NAVMAT  Instruction  (NAVMATINST)  5460.2  (Washington,  D.C  Naval  Material  Command, 

20  August  1969 

'ONAVELEX  NOTICE  5430,  2  August  1972. 

11  Naval  Material  Command,  "Electronic  Test  Equipment.  Classification  and  Assignment,” 
NAVMATINST  5430.52  (Washington,  D  C.:  Naval  Material  Command,  10  May  1973);  imple¬ 
mented  by  Naval  Electronic  Systems  Command,  "Naval  Material  Command  Electronic  Test 
Equipment  Classification  Board;  Policies  and  Procedures,”  NAVELEX  Instruction  5420  12 
(Washington,  D  C.:  Naval  Electronic  Systems  Command,  26  October  1973)  (Revision  A,  dated 

21  April  1976). 

12MIL-STD-1387  (Navy),  "Procedures  for  the  Acquisition  of  Non-Standard  General  Purpose 
Electronic  Test  Equipment,”  first  issued  in  1974  and  cited  in  NAVMATINST  5430.52,  Change  1, 
24  July  1975.  Specific  procedures  are  set  forth  in  NAVELEX  Instruction  5450  29,  "Navy  General 
Purpose  Electronic  Test  Equipment  (GPETE).  Policies,  Procedures,  and  Responsibilities  for,” 
24  June  1974 
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Thus,  since  the  mid- 1970’s,  the  Navy  has  had  the  policy,  organization,  and  procedures  in 
place  for  a  "single  manager”  of  GPETE  The  process  appears  to  be  controlling  unnecessary  prolifera¬ 
tion  and  achieving  economies-of-scale  in  GPETE  procurements  (subject  to  procurement  regulations 
that  favor  competition,  making  sole  source  reprocurement  and  multiyear  procurements  difficult) 
MIL-STD-1364  has  been  revised  every  2  or  3  years  (the  current  version,  M1L-STD-1364F,  is  dated 
March  1982),  with  interim  quarterly  updates  (entitled  "GPETE  Status  List  with  Reference  Prices”). 
Updates  consist  of  additions  to,  as  well  as  deletions  from,  the  preferred  list  of  GPETE  in  MIL-STD- 
1364.  The  net  result  is  that  the  number  of  line  items  (makes/models)  on  the  preferred  list  for 
standard  GPETE  remains  between  300  and  400  items. 

In  turn,  MIL-STD-1364  updates  are  reflected  in  the  standard  item  lists  maintained  by 
the  Naval  Sea  Systems  Command  (NAVSEA)  (cognizant  office  SEA-06C1,  Test  and  Monitoring 
Systems  Division)  and  Naval  Air  Systems  Command  (N'AVAIR)  (cognizant  offices:  AIR-552,  Support 
Equipment  Division,  and  Naval  Air  Engineering  Center,  Lakehurst,  New  Jersey)  for  the  purposes  of 
standardization  and  inventory  control.13  Those  lists  show  the  makes/models  of  ETE  (GPETE  as  well 
as  SPETE)  and  ancillary  items  in  the  actual  inventory.  They  are  organized  by  test  equipment  type 
(subcategory  code)  and  show  for  each  type  the  specific  makes/models  in  the  inventory,  with  the  first 
one  listed  being  the  specific  make/model  (if  any)  from  the  MIL-STD-1364  latest  update  When  new 
weapons  systems  are  developed,  contractors  are  required  to  screen  MIL-STD-1364  and  the  standard 
item  list  (in  that  order)  to  determine  whether  test  equipment  requirements  can  be  met  with 
equipment  that  is  already  in  the  inventory.  If  that  is  the  case,  then  the  contractor’s  SERD  will  iden¬ 
tify  the  preferred  make/model;  if  the  requirements  cannot  be  met  by  existing  test  equipment,  then  the 
SERD  will  identify  what  is  needed.  In  the  latter  case,  following  validation  by  the  Navy  (office 
symbols  SEA-06C1  or  AIR-552)  and  revision  of  the  test  equipment  allowance  list  for  the  units 


I3Naval  Sea  Systems  Command,  "Test  Equipment  Index  To  Shipboard  Portable  Electronic 
Test  Equipment  Requirements  List  (SPETERL),”  NAVSEA-0967-LP  008  9000,  and  Naval  Air 
Systems  Command,  "Avionics  Preferred  Common  Ground  Support  Equipment  Index,”  N  AVAIR  16- 
1 -525  (Washington,  DC.:  Department  of  the  Navy,  undated). 


receiving/supporting  the  new  weapons  system,  the  process  eventually  updates  the  standard  item  list 
with  a  new  type  ofGPETE  and  submits  a  procurement  request  to  NAVELEX.  NAVELEX,  supported 
by  the  Naval  Electronics  System  Engineering  Activity,  St.  Inigoes,  Maryland,  again  evaluates  the 
need  for  this  nonstandard  GPETE,  and  if  it  approves  the  need,  it  includes  the  requirement  in  its  bud¬ 
get.  The  test  equipment  is  actually  purchased  by  the  Ships’  Parts  Control  Center,  Mechanicsburg, 
Pennsylvania. 

As  a  result  of  evolving  technology,  the  number  of  different  test  equipment  types  within 
the  Navy  has  been  growing.  For  example,  NAVSEA  currently  has  approximately  550  subcategory 
codes,  including  140  for  standard  GPETE  and  a  similar  number  for  nonstandard  GPETE  The 
remaining  codes  are  categorized  as  both  SPETE  and  ancillary  items.  The  most  recent  subcategory 
codes  may  have  1  or  2  line  items  in  the  inventory,  while  the  older  subcategory  codes  may  have  10  to 
15  different  line  items.  NAVAIR  has  a  similar  distribution,  with  its  150  different  types  of  GPETE. 

With  respect  to  the  effectiveness  of  the  entire  process,  NAVELEX  may  have  gone  too  far 
with  keeping  MIL-STD-1364  up  to  date.  The  traditional  problem  with  PILs  is  that  they  tend  to  get 
out  of  date  and  thus  lose  their  utility  NAVELEX’s  quarterly  updates,  on  the  other  hand,  may  be  too 
frequent,  causing  instability  in  procurement  programs  and  contributing  to  some  proliferation  of 
GPETE.  Specifically,  NAVAIR  notes  that  the  TMDE  allowance  list  for  Aircraft  Intermediate  Main¬ 
tenance  Departments  has  about  doubled  over  the  past  14  years.  Part  of  the  growth  can  be  attributed 
to  evolving  technology,  but  part  may  be  attributed  to  the  strict  adherence  to  MIL-STD  1364  and  its 
frequent  updates. 

Other  initiatives  by  NAVELEX  in  GPETE  management  include  the  establishment  of  a 
"GPETE  Assets  Screening  Pool”  program,  a  central  registry  of  Fleet  assets  in  excess  of  allowance  for 
redistribution.  A  1980  GAO  report  noted  that  such  a  capability  was  lacking  in  the  Navy,  with  the 
exception  of  a  NAVSEA  program,  created  in  1977. 14  That  program,  which  is  being  transferred  to  the 

14U  S.  General  Accounting  Office,  "Survey  of  DoD’s  Management  of  Automatic  and  General 
Purpose  Electronic  Test  Equipment  (LCD-80-106),”  Letter  Report  to  the  Secretary  of  Defense, 
B- 199353  (Washington,  D  C.  U  S  General  Accounting  Office,  4  September  1980) 
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Fleet,  has  increased  the  availability  of  test  equipment  More  recently,  NAVELEX  has  been 
installing  an  automated  GPETE  data  base;  the  first  version,  excerpted  in  mid- 1984  from  the  more 
comprehensive  MEASURE  (Metrology  Automated  System  for  Uniform  Recall  and  Reporting)  data 
base,  is  incomplete  but  will  be  enhanced  to  include  all  GPETE. 15 

In  summary,  the  Navy  appears  to  have  an  effective  management  process  to  control 
proliferation  of  GPETE  insofar  as  current  procurement  regulations  permit  GPETE,  however,  is  only 
part  of  the  manual  ETE  inventory;  SPETE,  together  with  both  "limited  availability”  and  "limited 
demand”  GPETE,  account  for  about  half  of  the  ETE  inventory  and  escape  centralized  management. 
We  do  not  know  the  extent  to  which  the  SPETE  category  includes  items  that  could  or  should  be 
classified  as  GPETE.  Furthermore,  the  Navy  has  not  assigned  a  high  priority  for  replacement  of  its 
test  equipment.  As  a  result,  a  significant  portion  of  today’s  GPETE  inventory  (550,000  items  valued 
at  $2  billion)  is  obsolete,  i.e.,  out  of  production  and  not  economically  supportable  NAVELEX  has 
estimated  that  13  percent  of  its  GPETE  inventory  is  obsolete,  which  implies  a  funding  requirement  of 
about  $250  million  for  wholesale  replacement.  While  the  Navy’s  annual  funding  level  for  GPETE 
procurement  has  increased  in  recent  years  (from  $7  million  per  year  in  the  late  1970’s  to  $60  million 
for  1985),  it  is  still  insufficient  to  support  planned  replacements  even  if  there  were  no  new  require¬ 
ments.  The  average  economic  life  of  ETE  is  commonly  estimated  at  8  to  10  years,  implying  an  annual 
funding  requirement  of  $200  million  to  $250  million  just  to  avoid  obsolescence. 

A  recent  Navy  review  of  the  integrated  logistic  support  (ILS)  audits  for  28  weapons 
system  acquisition  programs  shows  that  the  Navy  is  still  experiencing  problems  with  support  and  test 
equipment  in  the  acquisition  process  [see:  F  A  Myers,  et  al  .  NAVSEA  ILS  Audits  Analyses  And 


15MEASURE  is  a  Navy-wide  management  information  system  in  support  of  the  Navy's 
metrology  and  calibration  program.  Its  primary  functions  are  to  1 1 )  inventory/recall/schedule 
metrology  assets,  (2)  document  calibration  actions,  and  (3)  collect  engineering  data  for  monitoring 
effectiveness  of  calibration  and  determining  calibration  intervals.  Navy  wide  implementation  of 
MEASURE  commenced  in  1975 


Lessons  Learned  (Bethesda,  Maryland:  David  W.  Taylor  Naval  Ship  Research  and  Development 
Center,  June  1983)1.  That  review  noted  the  following  significant  deficiencies: 

•  In  7  of  28  programs,  the  calibration  requirements  and  procedures  for  ETE  had  not 
been  established  as  prescribed  by  policy.  In  six  cases,  calibration  requirements  had 
not  been  addressed  at  all;  in  one  case,  where  the  ETE  did  not  require  calibration,  this 
fact  was  not  indicated  in  the  ILS  plan. 

•  In  4  of  28  programs,  the  ILS  plan  was  inadequate  with  respect  to  test  equipment, 
including:  failure  to  identify  needed  ETE,  failure  to  include  a  maintenance  plan  for 
the  prime  equipment,  failure  to  indicate  no  test  equipment  is  needed  because  of 
vendor  maintenance  of  the  prime  equipment,  and  failure  to  indicate  that  selected 
GPETE  is  in  critically  short  supply 

•  In  3  of  28  programs,  the  stated  GPETE  requirements  were  not  in  accord  with  MIL- 
STD-1364  and  MIL-STD-1387.  The  selected  GPETE  either  omitted  preferred  items 
listed  in  MIL-STD-1364  or  included  items  not  on  that  list  without  prior  approval 
having  been  sought  for  such  nonstandard  GPETE  per  MIL-STD-1387 

Automatic  Test  Equipment 

Navy  initiatives  with  regard  to  automatic  testing  date  back  to  the  1960’s,  when 
NAVAIR  adopted  the  concept  of  general-purpose  ATE  for  avionics  support  aboard  carriers  A 
program  management  office  (PMA-238)  was  established  for  the  development  and  acquisition  of  a 
militarized  ATE  —  the  Versatile  Avionics  Shop  Tester  (VAST)  —  which  was  to  be  capable  of  testing  the 
weapon  and  shop  replaceable  assemblies  of  the  carrier-based  aircraft  under  development  at  that  time 
(F-14A,  S-3A,  and  E-2C).  The  first  production  contract  was  awarded  in  1968  and  the  first  VAST 
(nomenclature  AN/USM-247)  was  delivered  in  1972.  The  Navy  bought  a  total  of  95  V  AST  stations  at 
an  aggregate  acquisition  cost  of  $1  billion  (excluding  application  software  —  i.e. ,  TPSs  —  and  support 
software),  with  the  last  station  delivered  in  1978.  They  are  installed  aboard  carriers  and  at  Nava!  Air 
Stations,  Naval  Air  Rework  Facilities,  Naval  Air  Maintenance  Training  Detachments,  and  contrac 
tor  facilities  (for  TPS  development  and/or  productive  work).  VAST  is  the  largest  ATE  in  the  DoD 
inventory,  comprising  ten  bays  of  hardware. 

The  Navy’s  experience  with  the  VAST  program  showed  that  a  single,  general-purpose 
ATE  for  testing  a  wide  mix  of  assemblies  and  modules  was  not  very  practical.  Because  of  low 
throughput,  additional  VAST  stations  had  to  be  added  aboard  carriers  (the  original  plan  was  two  per 
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carrier),  and  additional  ATE  had  to  be  procured  to  meet  the  testing  workload  created  by  peacetime 
flying  hours.  In  1975,  in  response  to  widespread  problems  in  the  Fleet  with  ATE  and  TPSs,  the 
Assistant  Secretary  of  the  Navy  for  Research  and  Development  requested  that  a  quick  study  be  con¬ 
ducted  to  define  the  problems  and  develop  solutions  That  study  was  conducted  by  representatives 
from  NAVMAT,  Systems  Commands,  field  activities,  the  Fleet,  and  industry  consultants.  The  result¬ 
ing  report,  known  as  the  "Marcy  Report,"16  identified  the  problems  and  the  Navy  actions  required  to 
solve  them  (see  problem/solution  matrix  in  Figure  5-2).  Its  recommendations  included  the  following: 

•  Enforce  existing  Navy  policy  in  weapons  system  acquisition  regarding  support- 
ability 

•  Establish  a  central  ATE  management  group  within  NAVMAT,  reporting  directly  to 
the  Chief  of  Naval  Material  and  supported  by  comparable  organizations  within  the 
Systems  Commands. 

•  Educate  management  personnel  in  the  technical  and  management  issues  involved  in 
weapons  system  acquisition,  including  the  practical  problems  of  BIT  and  off-line 
ATE  hardware/software. 

•  Provide  quick  relief  to  the  Fleet  by  (1)  initiating  engineering  changes  (reliability 
improvements)  for  high-failure  items  of  prime  equipment  as  well  as  ATE,  (2)  estab¬ 
lishing  "tiger-teams”  to  respond  to  ATE  problems,  (3)  developing  organic  test  pro¬ 
gramming  capabilities,  and  (4)  prohibiting  deployment  of  off  line  ATE  without  prior 
approval. 

•  Develop  a  new  family  of  general-purpose  ATE  and  institute  policy  requiring  Chief  of 
Naval  Material  approval  of  any  off-line  ATE  acquisition 

•  Initiate  and  support  both  a  short-range  and  long  term  research  and  development 
program  in  automatic  testing  technology  under  supervision  of  the  central  ATE 
management  office. 

To  supplement  this  effort,  the  Navy  also  requested  input  from  industry.  The  Industry  Ad  Hoc  ATE 
Project,  comprising  174  experts  from  member  companies  of  five  sponsoring  industry  associations,  was 


16ATE  Ad  Hoc  Working  Group,  Report  on  Navy  Issues  Concerning  Automatic  Test,  Moni 
toringand  Diagnostic  Systems  and  Equipment  (Washington,  DC  Assistant  Secretary  of  the  Navy 
for  Research  and  Development,  13  February  1976) 


5  12 


FIGURE  5-2.  NAVY  ATE  PROBLEM/SOI, UTION  MATRIX 
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chartered  in  November  1975.  The  project  published  its  final  report  in  April  1977. 17  (See  Appendix, 
Table  A-3  for  a  summary  of  recommendations.)  This  effort,  in  turn,  was  extended  into  the  Industry/ 
Joint  Services  Automatic  Test  Project  when  it  was  recognized  that  the  Navy’s  problems  in  exploiting 
automatic  testing  were  common  to  all  Military  Departments.  As  indicated  previously,  the  latter 
study  was  completed  in  June  1980,  with  most  of  the  recommendations  being  followed  up  by  joint 
working  groups  under  the  auspices  of  the  JLC  Panel  on  Automatic  Testing. 

The  Navy  began  implementing  many  of  the  "Marcy  Report”  recommended  actions  in 
1976.  A  Test  and  Monitoring  Systems  (TAMS)  Project  Office  (MAT-04T)  was  established  as  the 
central  Navy  focal  point  for  all  ATE-related  matters.  The  Project  Office  was  assigned  responsibility 
for  review  of  acquisition  projects,  conduct  of  a  coordinated  research  and  development  program  in  test¬ 
ing  technology,  development  of  policies/procedures/tools,  establishment  of  a  list  of  "approved  ATE” 
(with  prime  equipment  program  managers  required  to  select  from  that  list,  permitting  waivers  only  if 
justified),  and  coordination  of  TAMS  offices  in  the  subordinate  commands  In  the  1980  NAVMAT 
reorganization,  the  TAMS  Project  Office  was  transferred  to  NAVELEX  and  assigned  the  role  of  lead 
Systems  Command  Project  Office  in  TAMS-related  matters  (including  BIT,  off-line  ATE,  as  well  as 
manual  ETE  and  associated  support  requirements  such  as  metrology  and  calibration) 

The  evolution  toward  an  effective  management  structure  in  the  Navy  with  associated 
policies,  procedures,  and  tools  has  been  slow  For  example,  in  1978  the  TAMS  Project  Office  was 
assigned  responsibility  for  establishing  a  family  of  preferred  off-line  ATE  for  Navy-wide  use 
(NAVMATINST  3960.4B,  "Policy  and  Responsibility  for  Automatic  Testing,  Monitoring  and  Diag¬ 
nostic  Systems  and  Equipment,”  17  July  1978).  It  was  also  assigned  responsibility  for  matching 
projected  test  requirements  with  ATE  options,  with  preference  being  given  to  existing  ATE  (NAV¬ 
MATINST  3960. 9A,  "Automatic  Testing  Guides  for  Project/Acquisition  Managers;  Promulgation  of,” 

17Aerospace  Industries  Association  of  America,  Inc.,  eta)  ,  Report  of  Industry  Ad  Hoc 
Automatic  Test  Equipment  Project  for  the  Navy.  3  vols  (Washington,  D  C  Aerospace  Industries 
Association  of  America,  Inc.,  April  1977). 


19  September  1979)18.  However,  it  took  until  1981  to  standardize  the  ATE  selection  process 
(NAVMAT  P9407,  "Support  Equipment  Selection  Analysis  Guide,”  November  1981)  and  until  1982 
to  prescribe  application  of  this  process  and  publication  of  the  preferred  ATE  list  (NAVMAT- 
INST3960.il,  "Selection  of  Off-line  Automatic  Test  Equipment,  Policy  for,”  11  February  1982) 
Similarly,  while  the  requirement  for  a  Navy-wide  data  bank  of  ATE  was  firmly  established  as  early 
as  1974  (NAVMATINST  5230.8,  "Data  Banks  for  Automatic  Test,  Monitoring  and  Diagnostic  Sys¬ 
tems  and  Equipment,  Utilization  of,”  14  November  1974,  and  NAVMATINST  4440.46,  "Inventory  of 
Automatic  Test,  Monitoring,  and  Diagnostic  Systems  and  Equipment;  Request  for,”  10  December 
1974)  and  the  need  confirmed  by  numerous  ATE-related  studies,  the  Navy  has  still  not  proceeded 
beyond  a  manual  ATE  inventory  survey,19  although  plans  are  now  being  made  to  develop  an 
automated  management  information  system  and  a  consolidated  data  base.20  Overall,  however,  the 
Navy  is  tackling  the  issues.  Both  the  TAMS  Program  Report  (reviewed  annually  since  July  1981) 
and  the  Test  Technology  RDT&E  [Research.  Development.  Test,  and  Evaluation!  Plan  (also  updated 
annually)  appear  to  be  well  managed.  Of  the  recent  initiatives,  two  — one  in  NAVAIR  and  one  in 
N A VSE  A  —  are  described  below. 


18This  instruction  promulgated  three  of  the  JLC  Guides  listed  earlier  in  Figure  4-1:  "Auto¬ 
matic  Testing  Acquisition  Planning  Guide,”  "Built-In-Test  Design  Guide,”  and  "Selection  Guide  for 
Digital  Test  Program  Generation  Systems  ” 

t9The  most  recent  data  are  provided  in:  Automatic  Test  Equipment  Inventory  Survey.  TAMS 
Project  Office  (NAVELEX  08T),  30  September  1982.  This  survey  lists  the  testing  characteristics  of 
346 distinct  models  of  ATE.  Exact  counts  of  the  ATE  inventory  are  unknown,  reliability  and 
maintainability  characteristics  are  not  included;  and  data  on  the  performance  of  self-test  diagnostic- 
software  are  not  given 

20A  consolidated  data  base  is  included  in  the  proposed  NAVELEX  Instruction  3960  4C 
(draft).  The  planned  system,  being  developed  by  the  Fleet  Analysis  Center,  is  a  real  time,  menu- 
based  system  with  terminals  throughout  the  Navy.  The  preliminary  data  base  design  includes 
18  data  fields  relating  unit-under-test  information  (17  data  fields)  to  test  equipment  identification 
(ldata  field).  In  contrast,  consideration  of  an  ATE  management  information  system,  including 
ATE  technical  characteristics  and  utilization,  is  still  in  the  very  early  planning  stage,  see: 
Technical  Report  on  Test  Equipment  Data  Base  Investieation.  (Washington.  DC  Naval  Electronic 
Systems  Command,  August  1984). 


NAVAIR,  in  assessing  the  lessons  learned  from  VAST,  formulated  its  ATE  plans  for  the 
lLJO’s  and  beyond,  as  documented  in  the  "NAVAIR  ATE  Program  Plan”  (dated  January  1978, 
approved  and  promulgated  in  1979).  That  plan  focuses  on  achieving  seven  goals: 

•  Integrating  ATE  program  management 

•  Improving  ATE  acquisition 

•  Designing  avionics  for  testability  and  maintainability 

•  Minimizing  the  variety  of  ATE 

•  Consolidating  and  improving  ATE  software 

•  Improving  the  quality  of  TPSs 

•  Attaining  full  and  timely  organic  support  capability. 

The  key  element  of  this  plan  is  minimizing  the  variety  of  ATE,  with  the  ultimate 
objective  of  developing  an  ATE  inventory  with  standardized,  modular  hardware  and  software 
elements  using  a  single  standard  test  language.  To  achieve  that  goal,  the  plan  establishes  a  func¬ 
tional  family  of  common  ATE  to  be  used  in  the  1980’s  and  describes  a  new  ATE  project,  the 
Consolidated  Support  System  (CSS),  for  use  in  the  I990’s  and  beyond  Thus  far,  however,  NAVAIR’s 
instructions  on  the  selection  of  ground  support  equipment  have  not  been  updated  to  include  the 
concept  of  a  preferred  family  of  common  ATE.21  The  only  documents  expressing  this  policy  and 
listing  the  specific  ATE  involved  are  the  NAVAIR  Support  Equipment  Selection  Guide,  (dated 
June  1981)  and  a  Commander  Naval  Air  Systems  Command  Memorandum,  Subject:  "NAVAIR  ATE 
Policy”  (first  issued  on  12  June  1978  and  recently  reissued).  The  current  family  is  shown  in  Table  5-2. 
For  this  approach  to  be  successful,  however,  the  Navy  will  have  to  enforce  its  policy  to  standardize. 

21The  key  instruction,  NAVAIR  Instruction  5400  72,  "Policy  and  Responsibilities  for  the 
Selection,  Design,  Approval,  Ordering,  Delivery  and  Logistics  Support  of  GSE  (Ground  Support 
Equipment),”  20  June  1973,  is  strictly  neutral  with  respect  to  peculiar  versus  common  support 
equipment.  It  is  further  noteworthy  that  this  instruction  does  not  contain  a  single  reference  to 
pertinent  NAVMAT  instructions,  although  NAVAIR’s  family  of  ATE  is  a  subset  of  NAVMAT’s 
preferred  ATE  list. 


However,  as  indicated  by  the  JLC  AGSE  study,  that  standardization  will  be  very  difficult  because  of 


the  many  counterincentives  in  the  weapons  system  acquisition  process.22 

TABLE  5-2.  NAVAIR  FUNCTIONAL  FAMILY  OF  ATE 


YEAR 

NAME/NOMENCLATURE 

MFC.1 

TYPE 

INITIAL 

APPLICATION 

DATA 

PACKAGE 

VAST  ( AN/USM-247) 

Harris 

Digital- Analog- Hybrid 

Common 

Full 

1975 

CAT  IIID  (AN/USM-429  <  V)| 

Grumman 

Digital-Analog- Hybrid 

F  14A 

None 

1976 

1MUTS II  [AN/USM-608  ( Vi| 

Litton 

Inertial  Navigation 

F/A-18 

None 

1979 

EOSTS  [AN/AAM-60  ( V-6i| 

Boeing 

Electro-Optics 

S-3A 

In  Validation 

1980 

NEWTS  (AN/USM-458) 

Sanders 

Electronic  Warfare 

ALQ-126B 

None 

i m 

RADCOM  l  AN/USM-467) 

Grumman 

Radar 

E-2C 

None 

ATS  [AN/USM-470  ( V-l  )| 

Harris 

Digital-Analog  Hybrid 

F/A-18 

None 

US 

HTS(AN/USM-484) 

Harris 

Hybrid 

Common 

None 

1984 

ATS  (AN/USM-470  <  V-2)| 

Harris 

Digital- Analog- Hybrid 

SH-60 

None 

198X 

AEWTS(AN/USM-487) 

Honeywell 

Electronic  Warfare 

ASPJ 

None 

lMFG.  =  Manufacturer. 

SOURCE:  Program  Manager,  Consolidated  Support  System  iCSS),  "Navy’s  ATE  Project"  (Washington,  D.C.: 
Department  of  the  Navy,  1984  briefing). 


The  system  definition  phase  of  the  CSS  program  started  with  contract  awards  to  five 
competing  contractor  teams  in  January  1982  and  was  completed  in  August  1983  Navy  program  re¬ 
view  was  completed  in  March  1985  with  approval  to  proceed  into  full-scale  engineering  development, 
but  with  the  program  broadened  in  scope  to  include  the  intermediate-  and  depot-level 
ATE  requirements  of  the  other  Systems  Commands.  Renamed  the  Consolidated  Automated  Support 
System,  two  competitive  contracts  for  full-scale  development  were  awarded  to  General  Electric,  Incor¬ 
porated  and  Grumman  Aerospace  Corporation  in  September  1985.  NAVAIR’s  strategy  emphasizes 
supportability,  modularity,  technology  transparency  ( i.e. ,  designed  for  technology  updating),  rapid 
reconfigurability,  operational  evaluation  of  prototype,  and  ability  to  compete  the  production  phase 


22For  example,  while  the  nonradar  avionics  of  the  F/A-18  are  supported  by  the  Mini  VAST. 
AN/USM-470(V-1),  consisting  of  VAST  building  blocks  (50  percent),  new  design  test  equipment 
( 10  percent),  and  commercial  test  instruments  (40  percent),  the  AN/APG-65  radar  is  supported  by  a 
peculiar,  contractor-developed  radar  test  station,  the  design  of  which  does  not  include  any 
government  furnished  equipment. 


(leader/follower  concept),  with  deliveries  starting  in  1992.  Program  costs  are  estimated  at  $5.7  billion 
(fiscal  year  1983  dollars),  with  development  costing  $0.9  billion  and  production  $4.8  billion.  Com¬ 
pared  to  the  current  ATE  configuration  aboard  the  typical  carrier,  N'AVAIR  has  projected  the  follow¬ 
ing  savings  from  fielding  CSS: 

•  Fewer  test  equipment  types  (5  versus  95) 

•  Fewer  maintenance  technicians  in  the  intermediate-level  maintenance  shop 
(150  versus  250) 

•  Fewer  training  courses  (5  versus  185) 

•  Fewer  facility  space  ( 10,000  versus  15,000  square  feet) 

•  Fewer  contractor  technicians  (5  versus  21) 

•  Fewer  technical  publications  (45  versus  624  volumes) 

•  Fewer  spares  (3,800  versus  30,000  line  items) 

The  resulting  life  cycle  cost  savings  (compared  to  the  present  ATE  suite,  excluding  additional  costs 
for  needed  ATE  additions/replacements  to  support  future  weapons  systems)  have  been  estimated  at 
$3  billion 

NAVSEA's  initiatives  in  automatic  testing  are  comprised  under  a  single  program,  the 
Support  and  Test  Equipment  Engineering  Program  (STEEP).  Initiated  in  October  1978  under  the 
charter  of  the  Ship  Support  Improvement  Project,  STEEP’s  goal  was  to  improve  Fleet  support  of 
electronic  modules/PCBs.  The  program  entailed  a  cost  effectiveness  analysis  of  utilizing  ATE  (digital 
card  testers)  at  organizational-  and/or  intermediate-maintenance  levels  for  PCB  screening  and/or 
repair  versus  a  PCB  sparing  policy.23  An  evaluation  of  the  pilot  intermediate-level  installations 
using  commercial  digital  card  testers  and  TPSs  for  selected  PCBs  was  conducted  from  1979  to  1981. 
The  result  was  a  de  facto  change  in  maintenance  concept,  with  the  AN/USM-465  selected  as  standard 
intermediate-level  ATE  for  testing  digital  PCBs.  In  July  1981,  NAVSEA  fielded  the  first  operational 

23Since  1973,  Navy  maintenance  policy  for  electronics  equipment  has  consisted  of  organiza 
tional  replacement  of  modules/PCBs,  with  repairs  (if  not  throwaway)  allocated  to  intermediate  or 
depot  level  on  the  basis  of  level-of-repair  analysis  This  policy  resulted  in  a  two-level  maintenance 
concept,  with  PCB  repairs  at  depot  level 


TPSs  for  both  fault  detection  and  fault  isolation  at  intermediate-level  activities.  Additionally,  the 
cost  effectiveness  of  testing  at  the  organizational  level  was  evaluated  from  1981  to  1983  aboard 
DD-963  class  destroyers,  using  the  same  ATE/TPSs  planned  for  the  intermediate  level.  This  resulted 
in  the  decision  to  deploy  ATE/TPSs  aboard  selected  surface  combatants  for  selected  digital  PCBs. 

Between  July  1981  and  July  1984,  75  STEEP  sites  were  established  (51  shipboard, 
24  ashore),  with  100  AN/USM-465s  procured  and  deployed  and  about  430  unique  TPSs  developed  and 
installed  (total  density  10,500).  In  fiscal  year  1985,  STEEP  sites  are  scheduled  to  increase  to 
1 10,  AN/USM-465  inventory  to  several  hundred  (by  end  of  fiscal  year  1986  a  total  of  400),  and  unique 
TPSs  to  1,200  (total  density  some  20,000),  with  an  annual  growth  of  300  to  500  unique  TPSs  projected 
over  the  next  5  to  7  years.  To  manage  and  support  the  development  and  deployment  of  those  TPSs, 
STEEP  established  an  ATE/TPS  Coordination  Center  to:  (1)  provide  a  central  point  of  contact  for  all 
TPS  users,  (2)  maintain  ATE/TPS  configuration  and  deployment  status  accounting,  (3)  duplicate 
TPSs  for  deployment,  (4)  process  TPS  trouble  reports,  (5)  support  TPS  development  and  maintenance 
programming  tools  on-line  via  a  communication  network  to  TPS  developers,  and  (6)  provide  manage¬ 
ment  information.  The  Center  is  located  at  the  Fleet  Analysis  Center,  Corona,  California,  with 
computer  facilities  for  deployment  and  configuration  information  (operational  in  fiscal  year  1982) 
and  for  TPS  maintenance/development  tools  (operational  in  fiscal  year  1984).  The  communications 
network  is  illustrated  in  Figure  5-3.  Some  of  the  remote  sites  shown  in  that  figure  will  not  become 
operational  until  late  fiscal  year  1985. 

The  Center  distributes  quarterly  to  each  ship  the  Catalog  of  Automatic  Testing  Capabil¬ 
ity  for  Electronic  Modules/Printed  Circuit  Boards,  identifying  the  prime  equipment  and  the  specific 
modules  for  which  TPSs  are  currently  available  and  the  specific  test  sites  involved.  It  also  publishes 
the  Master  Test  Program  Set  Index,  listing  the  available  TPSs  by  ATE  type  and  weapons  system 
(While  the  AN/USM-465  was  selected  as  the  standard  ATE  for  STEEP,  other  card  testers  had 
previously  been  procured  by  NAVELEX  and  NAVSEA  in  support  of  selected  systems:  this  index  lists 
TPSs  available  for  four  ATEs:  AN/USM-465,  three  Phoenix  530  (3PX530),  Bendix  Herbie,  and 
AN/USM  422.) 
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FIGURE  5-3.  NAVSEAANAVELEX  ATE/TPS  COORDINATION  CENTER 
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SOURCE:  NAVSEA/N'AVELEX  ATE/TPS  Coordination  Center,  "Program  Status  Briefing’ 
(Corona,  California:  Combat  Systems  Assessment  Center,  July  1984) 

One  of  the  TPS  development  tools  that  is  supported  by  the  Center  is  the  Hierarchical 
Integrated  Test  Simulator  (HITS),  a  new  digital  Automated  Test  Program  Generator  (ATPG)  that 
was  developed  by  the  Naval  Air  Engineering  Center  with  contractor  support  (Grumman  Aerospace 
Corporation)  HITS  is  planned  as  the  standard  Navy  digital  ATPG  for  the  future:  it  provides  more 
capability  than  previous  ATPGs  especially  with  regard  to  large-  and  very-large-scale  integrated 
circuits 

Implementation  of  STEEP  is  providing  the  Fleet  with  a  much  needed  maintenance  capa¬ 
bility  Although  the  program  appears  to  be  well  planned  and  managed,  initially  it  suffered  from  the 


lack  of  coordination  between  NAVSEA  and  NAVELEX.24  That  problem  was  resolved,  however, 
when  NAVSEA  and  NAVELEX  signed  a  1984  memorandum  of  agreement,  with  NAVSEA  acting  as 
the  consolidated  management  activity  for  shipboard  ATE  and  both  commands  responsible  for  their 
own  TPSs,  using  jointly  developed  standards  and  procedures.  The  Fleet  still  has  not  become  familiar 
with  the  revised  maintenance  procedures  of  sending  PCBs  to  designated  intermediate-level  test  sites 
In  1983,  only  20  percent  of  Fleet  units  utilized  that  support  for  PCBs.  Finally,  the  development  and 
implementation  of  HITS  has  been  a  low-priority,  limited-funding  effort.  According  to  experts  in  the 
testing  community,  the  capabilities  of  HITS  are  modest  compared  to  such  proprietary  ATPG  systems 
as  LASAR  Version  6  (trademark  of  Teradyne,  Inc.)  in  which  hundreds  of  staff  years  have  been  invest¬ 
ed.  If  the  Navy  has  a  justified  need  for  its  own  ATPG,  it  will  have  to  increase  funding  priority  for 
enhancing  HITS 
AIR  FORCE  INITIATIVES 

The  Air  Force  has  undertaken  several  initiatives  to  improve  management  and  support  of  test 
equipment,  including 

•  Improving  management  structure,  policies,  and  procedures 

•  Establishing  the  Modular  Automatic  Test  Equipment  (MATE)  program 

•  Establishing  an  ATE  management  information  system 

•  Modernizing  precision  measurement  equipment  laboratory  (PMEL)  equipment 

•  Following-up  the  recently  completed  Support  Equipment  Acquisition  Review  Group  Study 

Management  Pol::icsand  Procedures 

In  the  late  1960’s,  AFLC  reorganized  its  management  structure  for  the  support  of  fielded 
weapons  systems  by  establishing  system  manager  and  item  manager  functions  at  its  Air  Logistics 
Centers  (ALCs)  The  item  manager  functions  were  assigned  as  follows 

•  Warner  Robins,  G leorgia:  avionics  and  electronic  warfare 

24See,  for  example  Naval  Audit  Service,  Repair  of  Shipboard  Electronics.  Audit 
Report  T  10582  (San  Diego,  California  Naval  Audit.  Service,  17  November  1983)  (U  S  v«overn 
ment  Only) 


•  Oklahoma  City,  Oklahoma:  engines 

•  Ogden,  Utah:  missile  and  munitions 

•  Sacramento,  California:  communications-electronics 

•  San  Antonio,  Texas:  test  equipment  (general  purpose) 

During  the  1970’s,  the  Air  Force  attempted  to  improve  its  management  of  support 
equipment,  especially  ATE,  because  of  problems  encountered  when  program  management  responsi 
bility  for  several  major  weapons  systems  (particularly,  the  F-lll  and  F-15  programs)  was  transferred 
from  the  materiel  developer,  AFSC,  to  the  command  responsible  for  life  cycle  support,  AFLC  The 
result  of  that  effort  was  the  establishment,  in  1982,  of  a  new  Deputy  Chief  of  Staff  for  Acquisition 
Logistics,  consolidating  AFSC’s  management  of  acquisition  logistics,  product  assurance,  standardiza¬ 
tion,  and  computer  resources,  with  the  Directorate  of  Policy  and  Programs  assuming  responsibility 
for  support  equipment  and  automatic  testing  policy  AFLC  established  the  single  system  manager  for 
automatic  test  systems  at  San  Antonio  ALC  (within  the  Directorate  of  Materiel  Management) 

The  Air  Force's  support  equipment  acquisition  process,  as  it  existed  in  the  early  1980’s, 
is  illustrated  in  Figure  5-4.  Concurrent  with  the  establishment  of  a  system  program  office  (SPO)  by 
AFSC,  AFLC  designates  the  ALC  to  be  the  System  Manager,  and  appoints  a  deputy  program 
manager  for  logistics  within  the  Air  Force  Acquisition  Logistics  Division,  a  field  organization  of 
AFLC,  acting  as  AFLC’s  arm  within  AFSC.  The  deputy  program  manager  for  logistics  heads  the 
AFLC  team  working  within  the  SPO.  The  SPO  is  also  supported  by  the  Support  Equipment  SPO  with 
its  personnel  represented  within  the  weapon  SPO.  The  process  depicted  in  Figure  5-4  is  the 
Air  Force’s  implementation  of  SISMS  (see  Chapter  4).  The  process  emphasizes  the  need  for  contrac¬ 
tors  to  screen  MIL-HDBK-300,  "Technical  Information  File  of  Support  Equipment”  (latest  version 
300M,  dated  1  October  1982),  which  is  maintained  by  the  AFLC’s  Cataloging  and  Standardization 
Office,  Battle  Creek,  Michigan.  SERD  review  by  the  Air  Force  is  extensive  and  involves  not  only  the 
SPO  and  System  Manager  but  also  the  AFLC  support  equipment  managers  (normally,  San  Antonio 


5  22 


Sj.J 

5  4  n  O 
"  J  5 

wiiii 


.5  =  2  s 

,  u  K  ^ 

5  *  *  5 


*  O  2 

I*  3  2  £> 
S3  5?2IS 


£  ii 

?*5 


,_>8!  ►  »  5  I  5 

5  “  o  t  ?  2  z  g 

S»  sj  z  z  o  <  £ 

2is  s  212“  u 

w  q  ifl  >Sr3S^—  2 

>£=!  ^  !d^u  5 

0<iu  a  ft-  <  O  <  o  <  < 

“  X  «JJ  j  3  U  u.  H  ~ 

_  .  t  •» _  £ 


d  d 

UJ  ^ 

CC  5  u,  * 

g  §  “  2 
>  5  u  o  *- 
P  w  S  x 


u.  vt  a. 

2  < 

o 


:  O  =  i  Jo  w; 

■  uj  5  a:  £  u  cr  - 
uj  £  uj 

I  I  >  =  K  I 


O  O  <  ^ 
£  z  2  g 
z  u  z  1 

uj  2  £  uj 

-  o  §  ta 

nS«u 

>  >  » I 
£Se8 

I  I  I 


I  « 


w  c 

?S“ 

u  v»  h- 


s  v  £  m  S 

!  5  g  s  I 

:  o  u  <  g 

:  £  *  t-  g  iu  O 

i  z  G  Z  ®  ¥>  Z 

;  >  55  d  K  o  < 

:  ^  O  i  ^  >  2 

;  £  S  2  “  g  ^ 

1  uj  uj  ui  J  Jr  u 


:  5  5  j  “{  2  J 
1  uj  uj  u  .  a  Q 

;  5  >  2  ^  <  “ 

!^gs8*2* 

I  .  .  £5  P 


s  K  w 
O  z  o 

yt 

SI  i 

UJ  w  Z 

®*o 

|e| 

*  *  5 

M  0  < 

?s  a 

2  g  o 

•  < 

“  ec 

o  “  5 
u  £  = 

>  <  s 


I.  „!  S  «  £  2 

I  «  2  £  8  £  <  2s;  , 

»“jS  J  s»?5s;s  i 

<  *  »  *  UJ  U  <  4  2  t  5  O  <J 

uj  f*  □  Q  •  “•  uj  T:  s  •  h 


2  S  UJ  C  . 

=  O  ff  O  ; 

u.  u  a.  uj  , 

UJ  UJ 

OKI  | 


-/  WSj^i 


SOURCK:  Joint  DARCOM/NMC/AFLC/AKSC  Commanders  Panel  on  Standardization  of  Aviation  (around  Support  Equipment, 
l'inal  Report  (Washington,  D  C  Joint  Logistics  Commanders,  28  June  1983). 


ALC),  the  ALC  responsible  for  depot-level  repair,  the  Aerospace  Guidance  and  Metrology  Center,  and 
the  Cataloging  and  Standardization  Office 


More  recently,  the  Air  Force  enhanced  the  role  of  acquisition  logistics  and  its  potential 
for  influencing  weapons  system  design  by  reorganizing  the  Air  Force  Acquisition  Logistics  Division 
as  a  joint  AFSC/AFLC  organization  designated  the  Air  Force  Acquisition  Logistics  Center.  As  we 
describe  in  a  later  subsection  (see  "Support  Equipment  Acquisition  Review  Group”),  the  Air  Force  is 
in  the  process  of  evaluating  further  improvements  in  management  policies  and  procedures  to  improve 
weapons  system  support  with  a  minimum  of  support  equipment  proliferation  and  better  support. 

MATE  Program 

Development  and  implementation  of  the  MATE  program  is  probably  the  most  important 
and  fundamental  initiative  undertaken  by  the  Air  Force  in  the  area  of  ATE.  Since  the  program  has 
been  well  publicized,  only  the  key  aspects  and  the  current  implementation  status  are  presented. 

The  MATE  program  was  established  in  1976  for  the  development  of  a  systematic 
approach  to  the  acquisition  of  ATE  required  by  future  weapons  systems.  The  approach  selected  was 
designed  to  overcome  the  ATE  problems  experienced  in  the  past:  proliferation  of  test  systems,  inade¬ 
quate  fault  detection  and  fault  isolation,  escalating  acquisition  costs,  shorter  life  cycles,  low  system 
availability,  technician  experience  drain,  and  a  lack  of  corporate  memory  of  lessons  learned  The 
Concept/Validation  phase  was  conducted  from  June  1978  to  June  1981,  with  two  competing  con¬ 
tractors  (Sperry  Corporation  and  Westinghouse  Corporation)  developing  a  uniform  ATE  architecture 
with  standard  interfaces  and  "intelligent”  (microprocessor-based)  stimulus/measurement  instru¬ 
ments.  The  product  of  this  phase,  then,  was  the  following  set  of  MATE  guides  providing  the  stan 
dards,  specifications,  and  procedures  for  each  contractor’s  concept: 

•  Electronic  Test  Equipment  Acquisition  Guide 

•  MATE  Development  Guide 

•  Avionics  Testability  Design  Guide 

•  Production  and  Operational  Support  Guide 

•  Test  Program  Set  Acquisition  Guide. 


In  July  1981,  the  full-scale  engineering  development  contract  ($55  million)  was  awarded 
to  Sperry  (the  selection  was  primarily  based  on  cost  because  the  two  concepts  were  similar).  The 
contract  required  refinement  of  the  MATE  guides,  development  of  the  MATE  data  base  system,  and 
demonstration  of  MATE  in  two  ways:  (1)  application  of  MATE  concepts  to  the  development  of  the 
intermediate  automatic  test  system  (IATS)  for  the  A-10  inertial  navigation  system,  with  Sperry 
responsible  for  ATE  integration;  and  (2)  technical  support  to  the  Air  Force  for  the  development  of  the 
Depot  Automatic  Test  System  for  Avionics  (DATSA),  replacing  the  obsolete  general-purpose 
automatic  test  system,  with  the  Air  Force  responsible  for  ATE  integration.  The  IATS  was  delivered 
in  August  1983,  and  a  formal  evaluation  was  completed  in  September  1984.  The  first  DATSA  was 
delivered  in  May  1984,  with  evaluation  continuing  through  May  1985. 

The  IATS  met  the  functional  qualification  test.  The  reliability  and  maintainability  test, 
reportedly,  indicated  it  did  not  meet  the  reliability  goal  (500  hours  mean  time  between  failures),  but 
did  meet  the  threshold  (200  hours).  The  Air  Force  has  exercised  its  option  with  Sperry  to  procure 
27  IATS  stations  (nomenclature  AN/GSM-294).  San  Antonio  ALC  is  now  developing  a  depot-level 
repair  capability  for  IATS  tester  replaceable  units  (TRUs)  and  modules  The  Air  Force  is  also 
acquiring  eight  DATSA  stations  under  contract  with  Emerson  Electric  (with  an  option  for  an  addit¬ 
ional  four)  for  installation  at  all  ALCs  for  repair  of  shop-replaceable  units  from  17  avionics  sub 
systems  of  the  F  -l  1 1,  C-141,  F-4,  F-105,  and  F-106  weapons  systems. 

The  MATE  program,  apart  from  its  importance  to  the  Air  Force,  had  a  significant  influ¬ 
ence  on  the  Army  and  Navy  because  it  encouraged  a  more  systematic  management  approach  to  the 
acquisition  and  support  of  ATE.  Major  portions  of  the  MATE  guides  are  under  review  for  adoption  as 
Joint  Service  guides  under  the  JLC  Panel  on  Automatic  Testing  (see  Chapter  4)  In  fact,  over 
50  percent  of  that  program’s  budget  spent  from  1978  through  1983  was  related  to  MATE. 


The  Air  Force’s  firm  commitment  to  MATE  is  demonstrated  by  its  publication  of  the 
implementing  regulation  in  early  1984.25  The  regulation  "applies  to  all  AFSC  and  AFLC  organi¬ 
zations  that  acquire,  modify,  replace,  and  support  Air  Force  systems  that  need  an  automatic  test 
system  (ATS)  for  logistic  support,”  but  is  not  retroactive  to  programs  that  have  completed  contract 
award  of  the  ATS  as  of  the  effective  date  of  the  regulation.  The  regulation  includes  the  policy, 
organizational  structure,  waiver  process,  and  authority  required  to  make  MATE  the  standard  way  of 
doing  business  for  ATE  acquisition.  It  implements  the  use  of  the  detailed  guides,  control  and  support 
software,  and  automated  tools  (Life  Cycle  Cost  Model,  TPS  Cost  Predictor  Model,  Software  Module 
Library,  MATE  Data  System,  and  ATLAS  test  program  evaluation  tools)  produced  by  the  MATE 
program.  It  establishes  the  functions  of  the  MATE  System  Policy/Control  Office  and  those  of  the 
Technical  Control  Agents.  It  spells  out  responsibilities  for  updating  the  MATE  System  Baseline  and 
the  MATE  Application  Baseline.  Finally,  it  directs  the  establishment  of  a  MATE  Operations  Center, 
installed  at  San  Antonio  ALC,  to  perform  MATE  qualification  testing  for  hardware  and  software 
modules  and  to  provide  access  to  the  MATE  automated  management  tools  and  training  system.  The 
Operations  Center  is  also  responsible  for  configuration  control,  maintenance,  and  distribution  of 
MATE  guides,  specifications,  standards,  and  other  documentation. 

The  key  factor  in  the  successful  implementation  of  MATE  has  been  the  cooperation  of 
the  test  equipment  industry  and  its  interest  in  meeting  MATE  qualification  standards.  The  suppliers 
of  commercial  test  equipment  had  to  commit  themselves  to  very  tight  requirements  in  the  DATSA 
program,  permitting  the  Air  Force  to  exercise  a  level  of  configuration  control  such  that  the  com¬ 
mercial  test  equipment  would  be  supportable  over  the  life  cycle  of  the  ATE  system.  Industry  partici¬ 
pation  in  this  program  appears  to  demonstrate  that  a  clear  and  well-reasoned  program  will  receive  a 
positive  response  from  test  equipment  manufacturers.  Some  well-known,  reputable  manufacturers, 
however,  have  announced  they  will  not  participate  in  the  MATE  program  because  their  standard 

25Air  Force  Systems  Command  and  Air  Force  Logistics  Command,  "Policy  for  Modular  Auto 
matic  Test  Equipment  (MATE),”  AFSC/AFLC  Regulation  800-23  (Washington,  D  C  :  Department, 
of  the  Air  Force,  25  January  1984) 


commercial  equipment  does  not  meet  MATE  specifications  and  would  be  too  costly  to  modify,  given 
the  lack  of  demand  for  such  specialized  equipment  from  the  commercial  sector. 

The  cost  benefits  of  MATE  are  difficult  to  express.  While  the  up-front  costs,  compared  to 
traditional  ATE  acquisition,  will  be  somewhat  higher,  life  cycle  costs  have  been  estimated  at  30  per¬ 
cent  less.  More  important  is  that  the  availability  and  effectiveness  of  ATE  can  be  expected  to  improve 
significantly  through  MATE  implementation.  On  the  other  hand,  the  risk  to  the  Air  Force  will  be 
somewhat  higher  because  it  assumes  integration  responsibility  for  the  test  system.  Past  experiences 
in  this  regard  have  been  negative,26  but  may  be  alleviated  by  the  MATE  guides  and  the  new 
management  structure  for  MATE  application.  Several  of  the  current  weapons  systems  in  develop¬ 
ment  or  in  major  modification  programs  are  not  utilizing  MATE  either  because  of  the  high  cost  (e  g., 
B-1B  program),  because  the  MATE  baseline  missed  the  program  window  (e  g.,  F-15  and  F-lll 
upgrade  programs),  or  because  of  commonality  (eg,  Peacekeeper). 

The  maintenance  concept  for  MATE  calls  for  on-station  test  instrument  replacement  at 
the  lowest  level  of  indenture,  which  normally  is  the  entire  TRU  for  commercial  instruments,  the 
shop-replaceable-unit  level  for  military  specified  instruments.27  The  concept  also  states  that 
commercial,  noncore  assets  are  returned  to  the  depot  or  the  manufacturer  (commercial  core  assets  are 
routed  via  PMEL)  One  would  thus  expect  that  the  question  of  where  to  repair  commercial  instru¬ 
ments— depot  or  manufacturer  — would  be  decided  on  a  case-by-case  basis,  using  traditional  level  of 
repair  analysis  However,  costs  were  not  considered  in  selecting  the  Air  Force's  maintenance  plan  for 
the  A- 10  inertial  navigation  system  I  ATS.  By  applying  a  standard  "decision  tree  analysis”  (in 


26For  example,  the  Air  Force  was  the  integrator  for  the  F-15  Tactical  Electronic  Warfare 
System  (TEWS)  Intermediate  Test  Equipment,  referred  to  as  TITE.  The  prime  contractor  developed 
the  ATE,  while  associate  contractors  in  the  TEWS  program  developed  the  test  requirements  The 
results  were  disappointing,  with  TPS  run  times  for  line-replaceable  units  up  to  100  hours,  requiring 
a  total  rework  with  the  prime  contractor  as  integrator  In  contrast,  the  F  16  program  used  the 
prime  contractor  for  development  and  integration  of  the  F-16  Avionics  Intermediate  Shop,  which 
was  successful. 

27See:  J.  Stout,  D  Persons,  and  J  Caporale,  "The  Air  Force  Modular  Automatic  Test  Equip¬ 
ment  (MATE)  Maintenance  Concepts,”  AL'TOTESTCON  '83  (New  York,  The  Institute  of  Electrical 
and  Electronics  Engineers,  Inc.,  1983),  pp  345-354 
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accordance  with  AFLC  Regulation  66-75),  which  does  not  consider  any  economic  aspects,  the  ATE 
manager  (San  Antonio  ALC)  arrived  at  the  recommendation  for  organic  (depot)  repair,  and  that  is  the 
approved  maintenance  plan  today.28 


ATE  Management  Information  System 

The  Air  Force  is  the  only  Military  Service  with  an  ATE  management  information  sys¬ 
tem  capable  of  providing  information  on  what  ATE  is  deployed,  where  it  is  deployed,  and  its 
utilization.  (The  Army  and  Navy  are  working  on  their  information  systems  but  do  not  plan  to  include 
any  information  on  ATE  utilization.)  A  brief  description  of  the  Air  Force’s  system  and  its  imple¬ 
mentation  status  follows. 

The  need  for  an  accurate  and  comprehensive  informational  data  base  was  formally 
identified  by  AFLC  in  1979.  The  "required  system  capability”  and  "data  automation  requirement” 
for  an  ATE  management  information  system  were  submitted  and  approved  in  1980  Development  of 
the  resulting  Test  Equipment  Reporting  and  Management  System  was  completed  in  1981,  and 
implementation  began  in  mid-1982  at  five  operational  air  bases.  The  system  consists  of  a  centralized 
data  base  (maintained  at  San  Antonio  ALC)  that  is  updated  on-line  from  the  participating  air  bases. 
The  input  data  include  ATE  utilization  records  (station  data,  unit-under-test  data,  and  test-run  data); 
ATE  status  records  (uptime/downtime,  reason  code,  TRU,  and  shop-replaceable-unit  data);  station 
inventory  records;  and  TRU  inventory  records.  The  system  is  designed  to  portray  trends  in  ATE 
operational  capability,  operational  time,  test  activity  and  results,  and  similar  management-oriented 
information.  The  implementation  of  the  system  worldwide  is  awaiting  reissuance  of  Air  Force  Regu¬ 
lation  65-1 10,  which  authorizes/directs  operating  commands  to  begin  feeding  input  data  to  the 
system.  Although  AFLC  has  repeatedly  requested  that  this  regulation  (or,  originally,  the  planned 
Air  Force  Regulation  65-4)  be  issued,  it  has  been  deferred  thus  far  because  of  reluctance  to  impose  yet 
another  reporting  burden  on  the  maintenance  work  force  However,  much  of  the  information  needed 


28Richard  L.  Shepherd,  "New  Start/Decision  Tree  Analysis  (DTA)  for  the  A  10  Intermediate 
Automatic  Test  System  (IATS),”  23  March  1984  (memorandum  and  supporting  justification  with 
attachments) 


for  a  suitable  management  information  system  capability  could  be  collected  automatically,  utilizing 
the  ATE  computer.  It  would  be  a  straightforward  exercise  to  add  a  software  module  to  the  ATE 
control  software  for  the  purpose  of  logging  all  ATE  transactions,  dumping  the  data  on  magnetic  tape 
or  disk,  and  mailing  or  transmitting  the  data  to  a  central  data  bank. 

San  Antonio  ALC  is  in  the  process  of  a  phased  enhancement  of  the  Test  Equipment  Re¬ 
porting  and  Management  System  to  interface  with  several  other  systems,  to  provide  major  commands 
access  to  the  system  (currently,  only  the  individual  bases  have  access  to  it),  and  to  include  several 
software  models.  A  total  of  $4  million  is  programmed  for  this  development  effort.  Ultimately,  the 
system  will  evolve  into  an  Air  Force-wide  reporting  system  with  the  installation  of  the  "Core  Auto¬ 
mated  Maintenance  System”  that  is  currently  under  development. 

PMEL  Equipment 

The  Air  Force  has  134  PMELs  that  use  8,000  measurement  standards  ("base  standards”) 
to  calibrate  some  900,000  items  of  test  equipment  of  55,000  different  kinds  (actually,  some  100,000  of 
these  900,000  items  are  calibrated  by  base  shops,  not  PMEL).  In  turn,  the  8,000  base  standards  are 
calibrated  against  215  Air  Force  standards  maintained  at  the  Air  Force’s  Measurement  Standards 
Laboratory,  which  is  operated  by  the  Aerospace  Guidance  and  Metrology  Center  (AGMC),  Newark, 
Ohio  29 

In  1980,  AGMC  found  that  the  PMELs  had  problems  performing  their  assigned  work¬ 
load  of  test  equipment  calibration  and  repair  and  faced  increasing  backlogs  and  reliance  on  contractor 
support.  In  1981,  AGMC  convened  the  first  worldwide  PMEL  conference  to  determine  the  causes  of 
those  shortfalls  and  the  actions  required  to  correct  them  It  found  that  the  four  main  problem  areas 
were  management,  manning,  training,  and  equipment  Following  that  conference,  the  Air  Staff  is¬ 
sued  a  Program  Management  Directive  to  develop  solutions  to  these  problems.  From  1981  through 

29Data  from  AGMC  briefing,  "PMEL  and  Sustainability,”  to  the  Deputy  Chief  of  Staff,  Logis¬ 
tics  and  Engineering,  early  1984 
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1984,  several  Air  Force  working  groups  developed  a  series  of  initiatives,  most  of  which  have  been 
implemented,  including  the  following: 

•  Policies/Plans:  Additions  were  made  to  the  War  and  Mobilization  Plan,  requiring 
planning  for  intermediate  maintenance  support  of  test  equipment  (previously,  the 
assumption  had  been  that  the  nearest  PMEL  could  provide  support,  not  recognizing 
that  all  PMELs  are  different).  Base-level  instructions  were  developed  to  identify 
PMEL  requirements  in  Operations  Plans  and  Support  Agreements.  A  proposal  (still 
under  consideration)  was  developed  to  expand  unit  status  reporting  to  include  PMEL 
shortfalls.  The  semiannual  PMEL  report,  submitted  to  the  Air  Staff,  was  expanded 
to  include  identified  PMEL  shortfalls  by  weapons  system.  Plans  were  prepared  for 
improving  deployed  PMEL  support  through  development  of  a  "tactical  PMEL,” 
upgrading  of  transportable  field  calibration  units,  and  adding  PMEL  support  to  the 
Combat  Logistics  Support  Squadron’s  mission. 

•  Equipment:  Most  of  the  PMEL  equipment  was  found  to  be  obsolete  because  of 
inadequate  funding  in  the  past.  A  significant  increase  in  funding  for  the  Metrology 
and  Calibration  (METCAL)  program  was  scheduled,  including  replacements  of 
obsolete  base  standards,  additives  for  new  systems,  and  some  research  and 
development  funding.  The  latter  was  necessary  because  the  National  Bureau  of 
Standards  no  longer  is  funded  to  support  DoD  research  and  development  require¬ 
ments.  Figure  5-5  illustrates  the  funding  profile. 

•  Procedures:  Procedures  were  developed  to  improve  item  management  of  PMEL 
standards  and  to  reduce  the  amount  of  test  equipment  fielded  without  calibration 
requirements  documentation.  A  PMEL  customer  complaint  process  was  established 
The  PMEL  Table  of  Allowance  was  reviewed  (for  the  first  time)  AGMC-prepared 
calibration  procedures  were  reorganized  (reducing  the  PMEL  calibration  procedure 
library  from  9,000  to  400  procedures)  and  rewritten  (reducing  technical  order  defi 
ciency  reports  by  35  percent)  and  the  procedure  verification  process  was  tightened 
The  process  for  distributing  technical  order  corrections  was  improved  (reducing 
response  time  from  6  months  to  30  days),  and  plans  were  developed  for  the  auto¬ 
mation  of  procedure  preparation  and  distribution. 

•  Personnel/T raining:  Proposals  (still  under  consideration)  were  developed  to  improve 
retention  and  increase  the  experience  level  of  PMEL  personnel  (in  the  early  1980’s, 
about  63  percent  of  the  PMEL  work  force  had  less  than  4  years  in  service).  A  new 
PMEL  manpower  standard  was  developed  and  evaluated.  An  occupational  analysis 
was  conducted  by  the  Occupational  Measurement  Center,  and  the  results  were 
reviewed  to  determine  changes  in  the  existing  formal  training  course  Current  plans 
are  to  develop  a  new,  career-oriented  training  approach  and  to  introduce  new 
training  technology  in  the  PMEL  school  at  Lowry  Air  Force  Base,  Denver.  Colorado 

Another  initiative,  undertaken  by  AGMC,  was  the  development  of  a  better  approach  to 
calibrating  ATE  Traditionally,  the  Air  Force  calibrated  ATE  in  a  similar  fashion  as  ETE.  i.e  ,  TRUs 
were  removed  from  the  ATE  at  specified  calibration  intervals  and  sent  to  the  appropriate  PMEL  for 
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FIGURE  5-5.  AIR  FORCE  METCAL  PROGRAM  FUNDING 


manual  calibration  It  was  recognized  that  this  "off-site  calibration”  of  ATE  has  many  disadvantages. 


including 


•  ATE  downtime  is  significant.  The  ATE  is  down  anytime  a  TRU  is  removed;  because 
the  TRUs  have  their  own  calibration  intervals  and  due  dates,  the  ATE  is  seldom 
completely  assembled 

•  Programmable  features  cannot  be  exercised  in  calibration  As  a  result,  timing  and 
time-related  problems  are  either  masked  or  ignored,  so  that  functional  TRUs  on  the 
test  bench  may  not  work  in  the  ATE 


System  performance  is  not  checked  No  compensation  or  allowance  for  cabling, 
loading,  switching,  or  other  sources  of  signal  degradation  between  the  TRU  and  the 
unit  under  test  is  possible  The  result,  again,  is  that  functional  TRUs  on  the  test 
bench  may  not  work  in  the  ATE  and  vice  versa. 


•  The  TRUs  are  subject  to  damage  in  transit  between  ATE  site  and  PMEL;  frequent 
removal  also  causes  unnecessary  wear  or  damage  of  connectors. 

•  Calibration  effort  is  wasted  on  unused  features  of  the  TRUs.  Very  few  ATE  applica¬ 
tions  use  every  feature  of  a  commercial  unit  and  even  fewer  use  a  feature  to  its  full 
rated  specification  (accuracy)  limits.  For  the  same  reason,  unnecessary  repair 
actions  may  be  conducted  on  features  not  used  by  the  ATE,  increasing  the  turn¬ 
around  time  of  TRUs 

•  High  levels  of  integration  in  modern  ATE  make  off-site  calibration  difficult  or 
impossible.  Many  measurement/stimulus  functions  previously  performed  by  indi¬ 
vidual  TRUs  are  now  often  implemented  at  the  card  level.  Removal  and  calibration 
of  those  functions  in  the  PMEL  is  difficult. 

•  The  TRUs  are  not  calibrated  in  their  operational,  physical,  and  electrical  environ¬ 
ment.  The  validity  of  off-site  calibration  is  thus  in  doubt. 

To  overcome  these  shortcomings,  AGMC  developed  an  on-site  calibration  concept,  the 
Portable  Automatic  Test  Equipment  Calibrator  (PATEC),  with  the  following  features. 

•  Portable  calibration  standards  are  brought  in  protective  cases  to  the  ATE  site 

•  The  ATE  is  calibrated  as  a  total  system,  addressing  only  the  required  functions  to 
required  accuracies. 

•  Calibration  is  performed  as  close  to  the  unit-under-test  interface  as  possible 

•  The  calibration  process  is  controlled  bv  a  program  on  the  embedded  computer  in  the 
ATE. 

•  Only  a  single  calibration  interval  applies  to  the  entire  ATE  station 

•  A  calibration  interface  test  adapter  is  used  when  necessary  to  provide  appropriate 
signal  conditioning,  but  it  does  not  contain  active  devices. 

•  PMEL  personnel  perform  the  ATE  calibration  with  user  (ATE  operators)  assistance 
and  participation. 

•  Most  PATEC  standards  are  programmable,  providing  as  totally  automated  a  cali¬ 
bration  procedure  as  possible. 

The  calibration  interface  test  adapter,  calibration  program,  and  associated  technical  documentation 
are  referred  to  as  a  calibration  test  program  set  (CTPS). 

From  1976  through  1984,  a  total  of  210  PATECs  were  bought  ($9  million  for  hardware), 
with  about  180  installed  or  to  be  installed  at  PMELs  and  the  remainder  to  be  used  for  CTPS 
development  by  contractors  for  selected  ATE  The  new  MATE  stations  ( A  - 1 0  1ATS  and  DATSA)  are 


among  the  ATE  for  which  CTPSs  are  being  developed.  AGMC  has  developed  briefings  for  ATE  con¬ 
tractors  and  guidelines/Data  Item  Descriptions  for  the  implementation  of  PATEC  for  new  ATE.  The 
benefits  of  PATEC,  in  removing  the  above-specified  shortfalls  of  off-site  calibration,  are  enormous. 
Neither  the  Army  nor  the  Navy  has  adopted  this  concept. 30 
Support  Equipment  Acquisition  Review  Group 

The  Support  Equipment  Acquisition  Review  Group  was  established  at  the  request  of  the 
Assistant  Secretary  of  the  Air  Force  (Research,  Development,  and  Logistics)  and  formally  chartered 
by  the  AFSC  and  AFLC  Commanders  in  March  1984  to  perform  a  study  of  the  entire  support  equip¬ 
ment  acquisition  process.  The  study  group  was  chaired  by  Major  General  Smith,  Commander, 
Air  Force  Acquisition  Logistics  Center,  with  participants  drawn  from  all  major  Air  Force  commands. 
The  study  was  completed  in  June  1984  with  briefings  to  the  Air  Force  Secretariat;  the  final  report 
was  published  in  July  1984  That  report  identified  19  major  problems  in  support  equipment  acquisi¬ 
tion  and  made  107  recommendations  to  solve  them  (see  Appendix,  Table  A-4  for  summary).  In  assess¬ 
ing  the  operational  impacts  of  the  problems,  it  concluded  that  shortages  in  support  equipment 
(estimated  at  a  cumulative  value  of  $15  billion)  do  not  impact  peacetime  mission  accomplishment 
because  of  extensive  work-around  procedures.  However,  those  shortages  could  potentially  have 


3°The  Army’s  EQUATE  is  calibrated  by  replacing  the  built-in  standards  at  specified  inter¬ 
vals  by  calibrated  spares  on  site,  with  the  replaced  standards  calibrated  off  site.  In  the  early  1970’s, 
the  Navy  developed  a  modular,  automated  calibration  concept  for  ETE,  known  as  MECCA  (Modu- 
larly  Equipped  and  Configured  Calibrators  and  Analyzers)  It  was  designed  as  a  portable  and  pro 
grammable  calibration  system  to  handle  80  percent  of  the  general  ETE  calibration  workload  It  is 
now  being  adopted  also  for  on-site  calibration  of  new  ATE  The  fundamental  distinction  with 
PATEC  is  that  MECCA  has  its  own  built-in  microcomputer;  PATEC  has  not,  but  uses  the  ATE 
computer 


mission  impacts  "if  we  are  stressed,  and  particularly  if  we  are  stressed  in  more  than  one  direction  at 
the  same  time.”31 

Among  the  many  recommendations  offered  by  the  study  group,  the  five  most  important 
ones,  according  to  a  separate  Air  Force  planning  document,32  are  the  following: 

•  Develop  and  maintain  a  master  plan  for  support  equipment 

•  Create  a  central  support  equipment  broker/advocate 

•  Treat  support  equipment  as  any  prime  equipment  program 

•  Include  support  equipment  considerations  in  early  program  planning  and  trade 
studies 

•  Program  the  funding  for  common  support  equipment  in  the  same  way  as  for  peculiar 
support  equipment. 

The  study  recommendations  have  been  approved,  in  principle,  by  the  Air  Force 
Secretariat,  and  the  Air  Staff  has  instituted  a  tracking  system  to  monitor  their  implementation. 
Some  of  the  recommendations,  such  as  the  creation  of  a  central  support  equipment  "broker,”  have 
reportedly  been  deferred.  Other  recommendations,  including  the  establishment  of  a  separate  pro¬ 
gram  element  for  common  support  equipment  and  the  development  of  an  ATE  replacement/ 
modification  roadmap  (as  a  subset  of  the  support  equipment  master  plan),  have  already  been  imple¬ 
mented.  Furthermore,  a  new  policy  for  more  effective  management  of  fielded  ATE  is  in  final 
coordination  (AFLC  Regulation  66-37,  "Management  of  Automated  Test  Systems”). 


3lQuoted  from:  Air  Force  Systems  Command  and  Air  Force  Logistics  Command,  Support 
Equipment  Acquisition  Review  Group:  Final  Report,  July  1984.  Note  that  the  report  defines 
support  equipment  as  comprising  two  categories:  TMDE  and  AGSE.  TMDE  is  defined  as  com¬ 
prising  precision  measurement  equipment,  ATS  (ATE  hardware,  ATE  software,  1  PSs),  and  special 
test  equipment  It  reports  the  support  equipment  inventory  to  consist  of  65,614  line  items  (NSN's), 
over  2  million  pieces  of  equipment,  with  a  total  cost  of  $9  billion.  This  is,  however,  the  centrally  pro¬ 
cured  inventory,  excluding  another  20,000  line  items  locally  procured  (unknown  cost,  unknown 
counts).  While  the  report  does  not  provide  a  breakout  between  TMDE  and  AGSE,  approximately 
80  percent  of  cost  and  line  items  pertain  to  TMDE,  and  the  quoted  cost  figures  refer  to  hardware,  not 
software  such  as  TPSs 

32See:  Thomas  E  Wenzel.  Avionics  Master  Plan,  ASD-TR-85-5001  ( Wright-Patterson 
Air  Force  Base,  Ohio.  Aeronautical  Systems  Division,  December  1984)  The  1985  edition  of  this 
annual  planning  document,  prepared  in  accordance  with  Air  Force  Regulation  800-28  ("Air  Force 
Policy  On  Avionics  Acquisition  and  Support”),  addresses  for  the  first  time  such  issues  as  standard¬ 
ization  and  avionics  support  equipment 
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APPENDIX 

SUMMARIES  OF  SELECTED  STUDIES 


This  appendix  summarizes  the  following  reports  in  tabular  format: 

•  Industry/Joint  Services  Automatic  Test  Project.  Final  Report,  June  1980  (Table  A-l) 

•  JLC  [Joint  Logistics  Commandersl  Panel  On  Automatic  Testing,  Subtask  Descriptions, 
30  September  1982  (Table  A-2). 

•  Report  of  Industry  Ad  Hoc  Automatic  Test  Equipment  Project  for  the  Navy.  April  1977 
(Table  A-3) 

•  Major  General  Monroe  T.  Smith,  Study  Chairman,  Support  Equipment  Acquisition  Review 
Group  Final  Report.  July  1984  (Table  A-4). 

The  table  entries  are  paraphrased  from  the  originals,  not  verbatim  quotes  Furthermore,  the  table 
entries  are  largely  limited  to  those  topics  germane  to  test  equipment  management  and  support.  A 
glossary  of  the  terms  used  may  be  found  at  the  end  of  the  Appendix 


AUTOMATIC  TEST  PROJECT 


AREA 

ISSUE 

RECOMMENDATION 

No  single  focal  point  is  responsible  for 
testability  requirements  and  evaluation 
during  weapons  system  acquisition,  thus 
precluding  effective  implementation  of 
testability 

Establish  a  central  point  of  responsibility  in 
each  Military  Service  for  testability  achieve¬ 
ment  in  weapons  system  acquisition. 

Organizations/ 

People/Funding 

Lack  of  funding  at  the  start  of  weapons 
system  development  for  purposes  of  ATE 
research  and  development  precludes  system 
planning  and  tradeoffs  for  cost-effective 
systems  support. 

Provide  ATE  funding  at  the  start  of  weapons 
system  development. 

Lack  of  development  of  advance  metrology 
standards  has  caused  universal  problems 
with  respect  to  traceability  and  repeatability 
that  continue  for  years  after  a  new  techno¬ 
logy  is  introduced. 

Encourage  the  Military  Services'  Metrology 
and  Calibration  Centers,  in  concert  with  the 
National  Bureau  of  Standards,  to  provide 
leadership  in  support  of  emerging  techno¬ 
logies. 

A  serious  communications  problem  exists 
with  respect  to  testability  because  of  the 
absence  of  universally  accepted  terminology. 

Develop  a  military  standard  on  definition  of 
testability  terms. 

Equipment 

Design 

The  proliferation  of  ATE.  excessive  support 
cost,  and  high  ATE  development  and  TPS 
costs  can  be  traced  to  poor  management  of 
the  ATE  acquisition  process  without  consid¬ 
eration  for  multiweapons  system  support. 

Promulgate  a  .Joint  Service  ATE  Acquisition 
Guide  that  embodies  specific  changes  in  the 
ATE  acquisition  process  as  identified  in  the 
Addendum  (see  page  A  ll  I. 

The  use  of  commercial  test  equipment  is 
increased  by  Government  directives  but 
cost-effective  application  of  this  equipment 
is  contingent  upon  adequate  technical 
documentation,  configuration  management, 
and  environmental  specifications. 

Make  specific  provisions  for  supportabilitv 
when  using  commercial  equipment 

Current  ATE  procurement  practices  recog¬ 
nize  the  existence  of  an  industry  ATE/ 
Instrument  Interface  Specification,  but  the 
frequent  use  of  alternative  or  additional 
control  and  data  interfaces  adds  to  the  pro¬ 
liferation  of  both  hardware  and  software 
associated  with  similar  ATE  requirements 
within  the  DoD 

Adopt  a  DnD-wide  ATE/Instrunient  Inter 
face  Specification  for  future  procurements, 
modeled  on  the  current  IEEE  Standard  4HH. 
but  updated  to  reflect  the  collective  input 
from  the  DoD  ATE  community. 

Specifications/ 

Directives 

Current  design  approaches  for  ATE  switch¬ 
ing  systems  tend  to  be  customized  to  a 
particular  application  such  that  modularity 
is  more  the  result  of  packaging  consider¬ 
ations  than  the  intentional  outcome  of  stan 
dard  functional  m  dulantv 

Adopt  a  DoD  wide  Interface  anil  Path 
Switching  Modularity  Specification  incorpo¬ 
rating  those  elements  deemed  necessary 
based  on  a  DoD  funded  studv. 

CDRL  data  items  supportive  of  MIL-STD- 
1 388  are  redundant  and  duplicate,  e  between 
items  and  between  the  Military  Services. 

Standardize  CDRL  data  items  across  the 
Military  Services  to  be  compatible  with  the 

MIL  STD  1 388  engineering  process. 

No  true  TRD  standard  exists.  Each  L’UT 
contract  imposes  some  tailored  version  of 
MIL-STD- 1 519  so  uniquely  applicable  to  one 
contract  that,  in  fact,  a  new  specification  is 
written  for  every  contract. 

Develop  a  common.  -Joint  Service  standard 
for  the  TRD. 

TABLE  A-l.  SELECTED  RECOMMENDATIONS  FROM  THE  INDUSTRY/JOINT  SERVICES 
AUTOMATIC  TEST  PROJECT  (CONTINUED) 


AREA 


ISSUE 


No  uniform  methodology  is  employed  in  the 
acquisition  of  ATE  software  incident  to  the 
acquisition  of  ATE.  Application  of  various 
existing  standards  regarding  performance 
requirements,  quality  assurance,  and  config¬ 
uration  management  is  largelyjudgmental. 


RECOMMENDATION 


Develop  a  military  standard  defining  the 
elements  of  ATE  software  and  establishing 
documentation  guidelines  to  support  re 
quirements  definition,  quality  assurance, 
configuration  management,  and  user  oper 
ation  and  maintenance. 


Specifications  / 
Directives 
(Continued) 


Repeated  formal  updates  of  ATE  data  items, 
including  TRDs  and  TPSs,  to  maintain  com¬ 
patibility  with  CUT  configurations,  gener¬ 
ate  significant  costs. 

The  marginal  quality  of  technical  publica¬ 
tions  creates  problems  at  all  intermediate- 
Ipvel  shops. 


Defer  ATE  data  item  delivery  until  weapons 
system  design  has  matured  and  stabilized. 


Improve  content  and  format  of  technical 
publications  for  the  ATE  operator,  CUT 
troubleshooter,  diagnostician,  and  ATE  re¬ 
pairman  through  additional  schematics, 
detailed  troubleshooting  information,  and 
better  formats.  Many  of  today's  ATE  short¬ 
comings  could  be  overcome  by  on-site  users  if 
they  were  given  adequate  information 


Most  ATE  application  software  exhibits 
serious  shortcomings:  1 1 )  TPSs  do  not  fa  ult- 
isolate  to  a  single  SRU,  take  too  long  to  run. 
and  do  not  distinguish  between  UUT,  ID, 
and  ATE  failures:  (2)  ATE  self-test  is  incom¬ 
plete,  unreliable,  and  takes  too  long  to  run; 
(3)  IDs  are  subject  to  failure  and  cannot  be 
checked:  (4)  too  many  different  languages 
are  in  use,  including  71  different  versions  of 
ATLAS,  with  few  usable  on  other  ATE:  and 
15)  the  program  test  philosophy  lacks  consis¬ 
tency. 

In  the  absence  of  accurate  mathematical 
models  of  analog  devices,  analog  ATPGs  will 
remain  infeasible  or  inadequate. 

Experience  with  digital  ATPGs  has  shown 
that  cost-effective  operation  depends  on 
certain  features  that  have  not  been  available 
in  the  past. 


Procure  test  software  with  specific  fault- 
isolation  requirements  and  verification  pro¬ 
visions  prior  to  fielding.  Provide  ATE  with 
the  capability  to  fault  isolate  to  the  UUT, 
ID,  or  ATE.  Provide  intermediate-level 
shops  with  capabilities  to  write/modify  TPSs 
subject  to  local  supervision  and  nonalterable 
Master  Programs.  Standardize  on  one  ATE 
language  and  impose  iton  all  future  procure¬ 
ments. 


Fund  the  development  and  demonstration  of 
several  analog  ATPG  systems  using  con¬ 
cepts  that  have  shown  promise  in  theoretical 
studies. 

Adopt  ATPG  guidelines  specifying  that: 

( lithe  ATPG  system  employ  a  dedicated 
general-purpose  computer.  (2ithe  ATPG 
system  be  interactive.  ( 3  >  the  simulator  be 


TPS  Development 
and  Management 


under  control  of  the  operator,  1 4)  logic  Ul  T 
diagrams  be  an  optional  output,  and  (5)  a 
combination  of  fault  dictionary  and  guided 
probe  methods  for  fault-isolation  be  avail 
able. 


Adequate  data  on  analog  faults  are  not  being 
collected,  leading  to  a  fruitless  debate  over 
the  necessity  of  detecting/isolating  soft 
faults  and  disregard  of  soil  faults  in  develop¬ 
ing  analog  ATPGs.  Soft  faults  are  fre 
quently  the  cause  of  hard  failures  of  the  next 
levelof  assembly 

With  increasing  1C  density,  the  complexity 
of  circuit  cards  will  outstrip  the  capability  of 
gate-level  ATPG  simulators  and  the  cost  of 
test  generation  w  ill  become  prohibitive 


Collect  statistical  data  on  hard  and  soft 
faults  in  analog  equipment.  Ensure  that 
analog  ATPGs  are  designed  to  detect/isolate 
hard  faults  in  the  presence  of  soft  faults 


Adopt  digital  ATPG  simulators  that  utilize 
both  functional  and  gate  level  models  to 
extend  their  capability  for  economical  test 
generation  to  LSI  and  VLSI  assemblies. 


£ 


AUTOMATIC  TEST  PROJECT  (CONTINUED 


AREA 

ISSUE 

RECOMMENDATION 

TPSs  have  not  lived  up  to  expectations,  often 
as  a  result  of  discovering  what  can  be 
afforded  at  a  late  point  in  the  total  system 
development  effort.  Both  TPS  development 
and  maintenance  costs  have  been  under¬ 
estimated. 

Establish  TPS  definition,  development  re¬ 
quirements.  schedule,  required  funding,  and 
life  cycle  cost  during  the  Concept  Formula¬ 
tion  phase  of  weapons  system  development. 

Many  TPSs.  although  they  perform  ade¬ 
quately  in  the  laboratory,  do  not  work  in  the 
user  environment  because  of  poor  quality. 

Issue  a  Joint  Service  TPS  Style/Design 
Guide  addressing  TPS  quality  aspects  in  the 
user  environment  (information  flow  to  oper¬ 
ator.  non-UUT  failures,  multiple-failure 
conditions,  etc.  i 

TP?  Development 
and  .Management 
(Continuedi 

The  definition  of  what  constitutes  a  TPS 
varies  among  designers,  suppliers,  buyers, 
and  users.  As  a  result,  the  TPS  invariably 
lacks  the  support  and  maintenance  data 
necessary  to  allow  the  user  to  make  neces¬ 
sary  TPS  modifications. 

Issue  a  Joint  Service  standard  defining  what 
constitutes  a  TPS.  It  should  encompass: 
object  code  media;  interface  device;  test 
program  instruction,  and  supporting  data 
including  UUT/interface  signal  identifi 
cation,  source  program  listing,  program 
flowchart,  program  description,  and  CUT 
schematics. 

The  lack  of  proper  standards  have  resulted 
in  a  high  risk  being  associated  with  TPS 
contracts  so  that  contractors  pad  their  bids 
Another  cause  of  excessive  cost  is  the  lack  of 
an  orderly  planned  approach  to  TPSdevelop 
ment  building  on  past  experience  or  results. 

Issue  the  Joint  Service  TPS  Development 
Planning  Guide.  emphasizing  a  phased  task 
approach  with  timely  design  reviews, 
quality  assurance  provisions  during  develop 
ment,  and  configuration  management  as 
pects  during  development  and  fielding 

The  lack  of  a  coordinated  family  of  ATE  for 
use  on  multiple  weapons  systems  results  in 
proliferation  of  specialized  equipment 
Without  standardizing  software  and  archi¬ 
tecture,  ATE  technology  will  be  moving  in 
many  different  directions,  precluding 
"technology  transparency" ol  future  ATE 

Fund  the  existing  military  programs  limed 
at  establishing  standard  families  ol  ATE 
i.MATE.  IFTE,  and  CSSi  but  ensure  thai 
these  programs  are  coordinated  and  that 
interface  and  software  issues  are  enipha 
sized 

ATE 

Current  ATE  systems  lack  uniformity  in 
language,  control,  display  arrangements, 
operator  messages,  and  procedures  as  a 
result  of  inadequate  hard w a r e; sol t  *  are 
guidelines 

Determine  common  user  information,  dis 
play,  and  control  requirements,  anti  develop 
hardware/soft,  .^re  guidelines  to  facilitate 
standardized  ATE  control  and  display 
arrangements 

Current  ATE  systems  accomplish  one  ra.sk  if  i 

a  time,  with  the  control  computer  and  .  t  ->  ! 

peripherals  idle  most  of  the  t  irne  Design  lor 
multinperat lonul  functions  would  increase  j 

ATE  cost  effectiveness  1 

Develop  ATE  hardware/soft w are  design 
guidelines  necessary  for  m ult  mper at  i*ma  1 
interfaces  using  distributed  processing 
•v\  '.terns  architectures 

No  industry  standards  exist  to  guide  the  1 
suppliers  of  ATE  interfaces  in  t  he  design  and  | 
use  of  a  programmable  interlace  pin  i  e  .  i 
L'l'T  interface  pm  capable  olbeing  sw  ih  bed 
under  program  control,  to  more  than  -me  * 

stimulus  signal  or  measurement  input  j 

1  "iidui  T  ,i  Dtil)  lunded  trade  studv  '•>  di-ter 
mine  .ippu«  ability  ■(  nheteiii  teafures  j 

on  iv  ersa  1  mt  ert.u  e  pm  li  •  t  he  dilVrent  main 
trna ni  e  lev  els 

TABLE  A- 1.  SELECTED  RECOMMENDATIONS  FROM  THE  INDUSTRY/JOINT  SERVICES 
AUTOMATIC  TEST  PROJECT  (CONTINUED) 


AREA 

ISSUE 

RECOMMENDATION 

ATE 

(Continued) 

Much  confusion  exists  on  subjects  related  to 
measurements,  calibration,  and  metrology. 
The  problem  is  common  to  industry  and 
Government,  is  of  domestic  and  inter¬ 
national  scope,  and  is  the  cause  of  misdirec¬ 
tion,  fragmented  responsibilities,  ineffective 
use  of  measurement  resources,  and  general 
inefficiency  in  metrology  and  calibration  for 
ATE. 

Support  the  NBS  in  development  of  a  series 
of  common  industry/Government  guides  for 
measurement  and  calibration:  ( 1 )  terms  and 
definitions,  (2)  specification  criteria  for 
generic  types  of  test  equipment,  and  (3 )  other 
areas. 

No  centralized  technology  forecasting  and 
assessment  activity  oriented  to  DoD  needs 
presently  exists.  Consequently,  industry 
does  not  know  which  technologies  will  be 
most  important  to  the  DoD.  when  they  will 
be  needed,  or  what  impacts  they  will  have. 

As  a  result,  costly  "shotgun1’  approaches  are 
used  in  developing  new  technologies  and 
their  applications. 

Establish  a  centralized  Technology  Fore¬ 
casting  and  Assessment  Activity  for  re¬ 
search  and  development. 

N'ew 

Technology 

Techniques  and  facilities  used  to  test,  align, 
and  handle  weapons  systems  and  their 
components  are  usually  not  operationally 
suitable  at  field-maintenance  levels.  Tech¬ 
nology  trends  point  toward  higher  mainte¬ 
nance  skill  level  requirements,  less  main¬ 
tainable  systems  except  in  laboratory 
environments,  more  equipment  damage  in 
the  field,  and  more  personnel  injuries. 

Sponsor  research  and  development  programs 
to  develop  effective  techniques  and  facilities 
that  satisfy  the  sensor  alignment,  handling, 
and  test  requirements  resulting  from  new 
technology  Program  objectives  are  to 
produce  fietdable  test  and  alignment  equip¬ 
ment;  to  develop  better  design  approaches 
for  protecting  equipment  and  personnel;  and 
to  improve  operational  availability  through 
better  testability  and  BIT  design. 

New  technologies  can  solve  current  pro¬ 
blems  as  well  as  cause  future  problems. 
Barriers  between  technological  fields  are 
inhibiting  effective  applications  of  new 
technology  to  ATE.  ATE  implemented  are 
generally  not  technology  innovators,  but 
rather  appliersof  available  technology 

Establish  a  DoD/Industry  ATE  Technology 
Advisory  Group  to  coordinate  ATE  testing 
technology  projects  and  to  promote  imple¬ 
mentation  of  innovative  ATE  technology  (in 
analogy  to  the  DoD/Industry  Manufacturing 
Technology  Advisory  Group) 

Inadequate  interface  techniques  and  test 
instrumentation  limit  the  performance, 
reliability,  and  testability  aspects  of  new 
technology  implementations. 

Undertake  a  research  and  development 
program  to  develop  system  interface  and 
instrumentation  techniques  for  exploiting 
new  technology  applications. 

IEEE  Standard  488  (1975)  represents  an 
excellent  basis  for  a  DoD  ATE/lnstrument 
Interface  specification,  but  current  trends 
toward  distributed  systems  raise  fundamen¬ 
tal  questions  regarding  data  rate  and  data 
word  size 

Initiate  study  program  to  determine  need  for 
an  update  to  IEEE  Standard  48H,  as  applied 
to  DoD  ATE/instrument  interfaces. 

System  Software 

ATE  system  software  development  is 
complex  and  requires  other  management 
and  technical  approaches  than  used  for  other 
DoD  software.  Attempts  to  reduce  ATE 
software  costs  fail  to  deal  with  the  under 
lying  causal  factors 

Adopt  a  clear,  formal,  and  rigorous  defini¬ 
tion  of  the  ATE  system  software  life  cycle. 
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TABLE  A-l.  SELECTED  RECOMMENDATIONS  FROM  THE  INDL'STRY/JOINT  SERVICES 


AUTOMATIC  TEST  PROJECT  (CONTINUED 


AREA 

ISSUE 

RECOMMENDATION 

ATE  system  software  development,  control, 
and  maintenance  has  been,  and  will  contin¬ 
ue  to  be,  a  labor-intensive  process.  Auto¬ 
mated  support  tools  are  technically  feasible, 
but  development  of  rehostable  tools  has  been 
inhibited  by  the  lack  of  universally 
acceptable  system  programming  languages. 

Reexamine  DoD  software  acquisition 
policies  with  respect  to  both  the  formal 
process  and  the  use  of  standard  high-order 
languages.  Promulgate  guidelines  for 
development,  configuration  management, 
and  maintenance  of  ATE  system  softw  are 

ATE  system  software  is  currently  provided 
in  many  different  languages,  reducing  the 
opportunity  for  rehostability  and  increasing 
the  maintenance  demands  on  the  eventual 
system  user  The  application  of  standard 
high-order  languages  to  ATE  system  soft¬ 
ware  is  far  behind  that  in  the  EDP  environ¬ 
ment. 

Limit  the  choices  for  high-order  languages 
used  to  implement  ATE  system  software. 

There  is  no  consistent  set  of  software  quality 
assurance  objectives,  policies,  or  procedures 
available  today.  MIL-S-52779.  "Software 

QA  Program  Requirements,"  is  not  gener¬ 
ally  applied  to  ATE  software. 

Develop  and  apply  guidelines  and  standards 
for  ATE  system  software  quality  assurance 
and  quality  management. 

System  Software 
(Continued) 

Certain  components  of  ATE  system  software 
and  tools  for  software  development  could  be 
standardized  as  Government-furnished  soft¬ 
ware  to  reduce  costs  without  stifling  cre¬ 
ativity.  but  the  tradeoffs  are  unknown. 
Widely  used  tools,  such  as  ATPGs,  are 
promising  candidates  for  Government- 
furnished  software. 

Define  and  establish  a  basis  on  which  to 
evaluate  the  cost-effectiveness  of 
Government-funded  development  of  poten¬ 
tially  large-use  ATE  system  software  tools  or 
components. 

There  are  no  tools  or  guidelines  for  measur 
ing  or  projecting  ATE  system  software 
development  and  application  costs,  which 
precludes  effective  program  planning 

Develop  tools  and  guidelines  for  ATE  system 
software  cost  measurement  and  prediction. 

Lack  of  a  DoDapproved  industry  managed 
standard  test  language  has  resulted  in  a 
Military  Service  disagreement  about  course 
of  action  and  precluded  industry  investment 
pending  clear  DoD  direction  The  uncon¬ 
trolled  language  proliferation  has  caused 
user  hardship  and  unnecessary  cost  esca 
lation 

Adopt  IEEE  ATLAS  as  the  standard  DoD 
wide  test  language,  with  evolutionary 
development  of  the  standard  managed  by  a 
formal  industry  standards  organization  (not 
the  DoD).  and  optional  use  of  specific  subsets 
of  { he  standard  for  specific  areas  of  testing 

Standardisation  of  ATE  system  and  support 
software  should  be  in  terms  of  interfaces,  not 
products,  as  the  latter  results  in  prolifer 
ation.  lack  of  competition,  and  technological 
obsolescence  If  the  Military  Services 
continue  to  ’’standardize*’  (formally  or 
de  facto)  on  products,  this  would  effectively 
compromise  the  entire  language  standard 
izatton  etTort. 

The  DoD  should  not  adopt  a  standard  test 
language  compiler  but  fm  us  its  standard¬ 
ization  efforts  on  interfaces,  with  the  highest 
payoff  estimated  to  be  in  standardizing  com 
niands.  protocol,  and  svstem  messages 
related  to  the  operator  ATE  interface. 

ATLAS  has  been  developed  as  a  voluntary 
industry  standard  over  the  past  12  years. 
Rapid  advancements  in  prime  equipment 
technology,  the  normal  lag  in  support 

Participate  in  the  correction  of  known  defi¬ 
ciencies  in  the  existing  ATLAS  standard 
documentation  iIEEE41b>  Continue  to 
support  research  and  development  infest 

TABLE  A-l.  SELECTED  RECOMMENDATIONS  FROM  THE  INDUSTRY/JOINT  SERVICES 
AUTOMATIC  TEST  PROJECT  (CONTINUED) 


AREA 

ISSUE 

RECOMMENDATION 

System  Software 
(Continued) 

technology,  and  the  voluntary  nature  of  the 
ATLAS  standard  group  have  created  a  gap 
between  what  is  needed  and  what  currently 
exists. 

languages  to  explore  new  testing  areas,  to 
improve  the  test  programming  features  of 
ATLAS,  and  to  track  ongoing  technical 
innovations. 

Calibration  of  analog  test  equipment  is  a 
time-consuming  process  requiring  highly 
skilled  personnel.  Application  of  automatic 
microprocessor  control  can  reduce  the  time  j 
and  skill  requirements. 

Explore  expanded  use  of  microprocessors  in 
calibration  of  test  equipment,  including 
analog  sensors,  and  develop  a  design  guide 
for  automatic  calibration 

Very  few  ATE  systems  have  been  specified 
as  a  composite  unit  with  capabilities  defined 
at  a  common  reference  point,  accounting  for 
derating  due  to  unit  response/settling  time 
or  for  enhancement  created  by  arbitrary 
accuracy  compensation  software  correction 
factors.  Standard  criteria  and  discipline  do 
not  exist.  Significant  problems  exist  with 
specification  criteria  for  time-dependent 
parameters,  measurement  techniques,  and 
measurement  standards  to  support  needed 
measurement  accuracies. 


Assess  existing  and  develop  new  universal 
criteria  and  discipline  to  ensure  that  ATE 
testing  capability  be  defined  by  a  system 
performance  specification  at  the  l'UT 
interface  connectors  and  be  characterized 
dynamically  as  a  function  of  time,  including 
the  contribution  of  software  to  capability 
enhancement.  Support  NBS  in  the  develop¬ 
ment  of  measurement  standards  and  tech 
mques  as  required  for  improved  accuracies. 


Metrology 

and 

Calibration 


One  element  to  be  defined  early  in  ILS 
planning  is  the  calibration  concept/plan  for 
organizational-,  intermediate  .  and  depot- 
level  test  equipment,  including  the  cali¬ 
bration  of  the  calibration  standards 
themselves  for  the  life  cycle  of  the  system. 
This  element  of  ILS  has  frequently  been 
ignored  resulting  in  (licont'nued  support 
problems  relative  to  calibration  require¬ 
ments  at  all  maintenance  levels.  <2l  mis¬ 
matches  in  technology  applications,  and 
(3)support  tasks  for  which  no  funds  have 
been  programmed 


Authorize  and  require  the  Metrology  and 
Calibration  Centers  to  participate  in  the 
LSA  process  during  early  ATE  definition 
and  support  planning. 


Industry  and  Government  do  not  have  a 
system  for  effecting  coordinated  efforts  in 
support  of  measurement  technology  in  a 
timely,  efficient,  and  inclusive  fashion  NBS 
attention  has  been  diverted  to  consumer  and 
safety  affairs  and  away  from  metrology  over 
the  past  10  vears  NBS  commitment  is 
necessary  to  recover  and  restore  the 
situation  to  what  it  should  be 


Accelerate  >upport  to  the  NBS  in  taking  a 
more  active  leadership  role  in  the  measure 
ment  technology  field  to  be  responsible  for: 

•  Technical  assessment  and  standard 
izalion  of  METCAL  terminology,  delim 
turns,  specification  criteria,  and  verifies!- 
tion  techniques  for  generic  tvpes  of  test 
equipment 

•  Establishment  and  documentation  ol  the 
highest 

•  Attainable  measurement  capabilities  in 
NBS  for  every  measurement  parameter 

•  Identification  and  development  ol  re¬ 
quired  measurement  capabilities  exceed 
mg  the  existing  ones 


lit* 


TABLE  A-l  SELECTED  RECOMMENDATIONS  FiiOM  THE  INDL'STRY/JOINT  SERVICES 
AUTOMATIC  TEST  PROJECT  (CONTIN  UED) 


AREA 


Metrology 

and 

Calibration 
(Continued ) 


Maintenance 

Shop 

Productivity 


ISSUE 


The  traditional  measurement  policy  stipu¬ 
lates  achievement  of  a  minimum  accuracy 
ratio  ( typically.  4: 1 )  between  each  step  in  the 
chain  from  prime  product  measurement  to 
the  highest  order  standard  in  combination 
with  worst  case  tolerancing  at  each  step  to 
establish  a  high  confidence  factor  of  mea¬ 
surement  integrity  A  higher  accuracy  ratio 
generally  implies  a  higher  equipment  cost 
for  an  assumed  but  unquantified  gain  in 
measurement  integrity.  This  traditional 
policy  has  no  technical  basis.  If  an  adequate 
and  predictable  confidence  factor  could  be 
achieved  with  lower  accuracy  ratios  lie., 
cheaper  equipment),  immediate  benefits 
would  result  in  supporting  existing  and  in 
planning  for  new  equipment. 

A  common  support  problem,  when  different 
test  equipments  are  used  at  the  different 
maintenance  levels,  is  the  prevalence  of  test 
inconsistencies  This  leads  to  prolonged 
back  and  forth  movement  of  prime  assem 
blies  through  maintenance  pipelines, 
increased  costs  for  additional  prime  equip 
ment  spares,  and  reduced  weapons  system 
availability.  Resolving  the  problem  fre¬ 
quently  requires  the  fielding  of  duplicate 
factory  test  equipment  at  intermediate 
shops.  A  better  solution  is  the  concept  of 
vertical  commonalty  that  provides  for  com¬ 
mon  test  equipment,  software,  and 
procedures  at  intermediate,  depot,  and  facto 
ry  levels  to  ensure  test  consistency 


RECOMMENDATION 


Conduct  a  coordinated  study  with  NBS. 
industry,  and  Government  participants  to 
assess  the  fundamental  measurement 
criteria  of  tolerancing  and  accuracy  ratio 
techniques,  including  alternatives  to 
existing  policy  such  as  "statistical  toler 
ancing  and  accuracy  ratio  budgeting," 
relative  to  establishing  realistic  prime 
equipment  specifications,  supporting  ATE 
capability  requirements,  and  the  resulting 
calibration  specifications  Promulgate  study 
results  in  the  form  of  guides  for  use  by 
industry  and  Government  to  assure  proper 
application  of  the  most  cost-effective  ca  libra 
tion  approach. 


Revise  DoD  policy  documents  and  standards 
relative  to  maintenance  planning  and  ATE 
acquisition  to  include  the  concept  of  ver  tical 
commonalty  in  weapons  system  support 
across  all  maintenance  echelons  as  a  recom¬ 
mended  approach. 


The  highest  rated  problem  in  field  surveys  is 
the  inadequate  skill  levels  of  ATE  operators 
and  maintenance  personnel  due  to  made 
quate  training,  inadequate  selection,  higher 
turnover,  motivational  problems,  and  very 
low  retention. 


Institute  a  proper  personnel  and  training 
system  for  ATE  personnel,  including 
selection  on  the  basis  of  ATE  related  apti 
tudes;  training  in  accordance  with  present 
ATE  capabilities,  maintenance  philosophy, 
and  task  requirements,  controlling  their 
assignments  to  ATE  organizations;  reducing 
non  ATE  related  job  diversions;  providing 
retention  mcentiv  es  such  as  professional  pay 
and  assignment  preferences;  and  ensuring 
career-  level  training  for  ATE  supervisors. 


Management  control  in  the  ATE  shop  is 
hindered  bv  the  lack  of  real  time  status, 
priority,  production,  manning,  inventory, 
performance  measurement,  and  other  data 
needed  for  efficient  operation  of  the  shop 
Various  management  control  systems  are  m 
use.  but  they  are  manual,  causing  excessive 


Develop  and  implement  a  standardized  real 
time  management  information  system  for  ail 
large  ATE  shops  The  svstem  should  collect 
IT  T  and  ATE  status  tracking  data,  using 
tile  available  ATE  computer  peripherals, 
and  generate  the  following  management 
information  throughput  backlog,  NORS 


TABLE  A-l  SELECTED  RECOMMENDATION'S  FROM  THE  INDUSTRY/JOINT  SERVICES 
AUTOMATIC  TEST  PROJECT  (CONTINUED) 


AREA 

ISSUE 

RECOMMENDATION 

Maintenance 

Shop 

Productivity 

(Continued) 

paperwork  associated  with  status  tracking. 
There  are  no  consistent  measures  of  ATE 
shop  performance 

Facilities  iboth  ships  and  buildings)  that 
house  ATE  systems  are  often  undersized 
because  of  poor  planning  causing  inefficient 
operations.  Another  problem  in  most 
installations  surveyed  is  a  lack  of  adequate, 
properly  regulated  and  filtered  electric 
power,  exclusively  available  to  the  ATE. 
These  conditions  result  in  loss  of  main¬ 
tenance  time,  inaccurate  test  results,  and 
sometimes  equipment  damage. 

Maintenance  performance  in  ATE  shops  is 
highly  inefficient.  The  causes  include: 

1 1 1  insufficient  quantities  of  spares  are 
provided  for  prime  equipment  and  for  the 
ATE  itself;  (2)  spare  modules  and  piece  parts 
received  through  the  supply  system  fre¬ 
quently  are  not  equivalent  in  form-fit 
function  to  original  parts;  ( 3 )  responsibility 
for  supply  support  is  not  controlled  or 
managed  by  the  ATE  shop  manager  causing 
delays  in  obtaining  replacement  parts;  and 
(4) storage  of  spare  parts  for  prime  equip¬ 
ment  and  ATE  is  often  remote  from  the 
repair  work  center  causing  loss  of  repair 
time. 

Existing  ATE  has  too  many  shortcomings 
including  the  following:  (litoo  much  ATE 
proliferation,  often  in  the  form  of  large, 
single  ported  stations;  <2l  too  many  IDs  that 
are  too  complex,  take  too  long  to  connect,  are 
not  people  proof,  are  subject  to  hookup 
errors,  and  come  with  bulky  cables  and 
fixtures  that  often  prevent  access  to  ID  or 
UUT  adjustment;  (3)  station  calibration  is  a 
nightmare,  taking  much  longer  than 
planned,  frequently  resulting  in  building 
blocks  that  will  not  work  when  reinserted 
alter  off  station  calibration,  and  degrading 
shop  throughput  capability;  <4l  ATE  is  too 
dependent  on  a  controlled  environment  and 
-.table  power  inconsistent  with  the  opera 
(tonal  environment;  and  ' 5 >  ATE  is  too  noisy, 
requiring  the  use  of  noise  muffs  or  acoustic 
curtains  and  frequently  resulting  in  pre 
mature  fatigue,  lausea,  and  decreased  efii 
ciencv  of  ATE  personnel 

and  NORM  rates  of  supported  systems, 
productivity,  ATE  availability,  and  ATE 
utilization  rate. 

Facilities  planning  for  ATE  should  include 
space  requirements  for  all  ATE  system 
components,  and  a  source  of  dedicated  power 
of  good  quality 

Establish  a  single  manager  for  both  the  end 
item  and  the  spares/piece  parts  to  support  it, 
including  those  for  the  supporting  ATE. 

Encourage  procuring  agencies  to  allow 
maximum  use  of  COTS  parts  for  ATE.  Place 
intermediate-level  supply  support  for  ATE 
under  the  authority  of  the  intermediate- 
level  ATE  manager  Colocate  repair  parts 
for  ATE  and  for  weapons  system  LRUs/ 
WRAs  in  the  same  physical  area  close  to  the 
repair  work  center 

Control  types  and  features  of  ATE  hardware, 
starting  with  the  procurement  of  a  family  of 
small,  generic  ATE  (digital,  RF.  servo,  etc.), 
and  impose  compatibility  requirements  on 
future  weapons  system  acquisitions.  Stan¬ 
dardize  the  physical  ATE  interface  to  the 

ID,  standardize  UUT  connector  families,  and 
include  minimization  of  the  numbers  and 
complexity  of  IDs  as  formal  evaluation  crite¬ 
ria  in  acquisition. 

Insist  that  future  ATE  be  self-calibrating, 
except  for  primary  standards:  with  respect  to 
existing  ATE.  provide  more  spares  for 
calibrating  components  and  calibration  in 
place,  where  possible.  Require  that  ATE 
operate  on  normal  line  power  and  be  imper 
vious  to  damage  by  voltage  transients  and 
spikes:  and  that  operating  temperatures  be 
consistent  with  intermediate  level  environ¬ 
mental  conditions  w  ithoul  external  support 
'cooling!. 

Consider  noise  abatement  when  procuring 
ATE  by  including  maximum  allowable 
acoustic  noise  in  procurement  specifications. 
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ADDENDUM  TO  TABLE  A-l 


SPECIFIC  CHANGES  IN  THE  ATE  ACQUISITION  PROCESS 
TO  BE  IMPLEMENTED  VIA  A  JOINT  SERVICE  ATE  ACQUISITION  GUIDE 


The  next-generation  family  of  ATE  needs  to  be  defined  via  a  well-managed  acquisition  process 
and  not  driven  by  ATE  technology  alone  as  has  happened  in  the  past  Specifically,  the  Military- 
Services  need  to  implement  a  formal  process  for  ATE  acquisition,  with  checkpoints  and  measures  to 
ensure  that  the  process  has  been  followed.  The  following  six  major  areas  have  an  impact  on  the  ATE 
acquisition  process  and  the  associated  suggestions  are  offered  for  implementation  through  a  Joint 
Service  Acquisition  Guide  for  Automatic  Test  Equipment. 

DATA  REQUIREMENTS/TECHNOLOGY  CENTERS 


The  data  requirements  that  are  essential  to  successful  acquisition  fall  into  two  categories:  data 
required  from  the  prime  system  designer  and  historical  data  required  to  exercise  the  tradeoff  tools 
used  by  management  in  the  decision  process.  The  following  documented  results  of  Logistic  Support 
Analysis  (LSA)  and  Test  Requirements  Analysis  (TRA)  should  be  provided  by  the  prime  contractor: 

•  Test  objectives  and  test  functions 

•  Level  of  repair:  on-line  and  off-line  (BIT  tradeoffs) 

•  Maintenance  sites  and  site-workload  calculations  from  reliability  data 

•  Source  documentation  suitable  for  programming  (TRD) 

Historical  data  are  required  to  support  the  tradeoff  tools  used  in  the  acquisition  process  These 
data  should  be  stored  in  technology  centers  on  a  Joint  Service  basis,  making  maximum  use  of  existing 
data  systems.  The  data  systems  should  be  broken  into  ATE  characteristics,  logistic  data,  and 
advanced  testing  technology  data,  with  appropriate  linkage  between  the  three 

Application  software  will  become  a  major  military  asset  and  should  be  integrated  by  the 
Military  Services  at  "software  centers,”  with  configuration  management  and  control  as  well  as  test 


program  generation  functions  included. 


UTILIZATION  OF  EXISTING  STATE-OF-THE-ART  CAPABILITY 

The  acquisition  process  should  call  out  as  a  recommended  approach  the  evaluation  of  inventory 
test  systems  and  subsystems  prior  to  procurement  of  new  or  modified  systems  The  test  system 
inventory  should  include  commercial  systems  for  those  applications  for  which  militarized  capability 
is  not  required  at  the  site  in  question.  As  new  systems  and  subsystems  are  developed,  they  should  be 
added  to  the  inventory  and  their  characteristics  entered  into  the  data  bank  to  encourage 
multiweapons  system  as  well  as  Joint  Service  utilization  of  this  capability.  The  test  equipment 
inventory  that  has  been  initiated  by  the  Military  Services  should  be  extended  to  include  inventory  of 
test  program  sets. 

IMPLEMENTATION  OF  SELECTED  STANDARDS 

Any  new  or  modified  test  systems  needed  must  be  procured  with  a  "standard  subset  of 
specifications”  determined  by  the  Government.  This  process  will  promote  lower  life  cycle  ATE  cost 
while  not  impairing  technical  progress.  (The  ATLAS  standard  language  is  the  first  such  specification 
agreed  to  by  DoD  components.)  Other  possible  future  specifications  may  include  standard  computer/ 
instrument  bus  interfaces,  and  standard  stimuli  or  measurement  devices  for  such  stable  technology 
areas  as  power  sources,  standard  input/output  controls,  and  displays 
SITE  WORKLOAD  ANALYSIS  REQUIREMENT 

This  process  requires  that  no  ATE  system  be  procured  without  a  level  of  repair  analysis  point 
ing  out  the  maintenance  level  and  site  at  which  the  ATE  will  be  utilized  Further,  for  each  site  for 
which  ATE  is  proposed  for  a  particular  weapons  system  and/or  equipment,  a  "site  workload  analysis" 
must  be  performed  In  addition,  wherever  a  site  supports  multiple  weapons  systems  and/or  equip¬ 
ment,  the  workload  analysis  should  include  all  workload  at  that  site  and  utilization  of  existing  ATE 
at  that  site  To  enhance  throughput,  the  use  of  large  systems  for  simple  tasks  should  be  avoided 
whenever  possible. 

The  complexity  of  the  test  system  is  directly  related  to  the  diversity  of  the  units  to  be  tested, 
therefore,  partitioning  workload  to  minimize  test  system  complexity  and  maximize  throughput 
should  be  a  goal  of  the  workload  analysis.  As  an  additional  requirement  of  site  workload  analysis,  a 
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phaseout  plan  for  obsolete  equipment  should  be  developed,  including  test  program  set  translation 
redevelopment,  or  discard. 

TESTABILITY  AND  ATE  TECHNOLOGY  DEFICIENCY  DOCUMENTATION  AND  FEEDBACK 
It  should  also  be  required  that,  as  a  result  of  the  above  analyses,  two  specific  deficiency  reports 
be  generated.  The  first  should  be  addressed  to  testability  problems  in  the  prime  system  design  and 
the  second  should  point  out  technology  deficiencies  in  state-of-the-art  ATE  The  technology 
deficiencies  will  provide  inputs  to  advanced  testing  technology  programs  so  that  all  ATE  procured  for 
production  programs  is  limited  to  off-the-shelf  technology 
ATE  SUPPORT  PLANNING  REQUIREMENT 

Finally,  the  process  should  require  that,  a  complete  ILS  plan  for  the  proposed  ATE  be  devel 
oped  The  ATE/ILS  plan  should  consider  the  specific  resource  constraints  (e  g  ,  supply,  personnel, 
space)  at  each  site  and  be  emphasized  in  all  procurements  and  tailored  to  that  application  ATE  self 
test  should  be  emphasized  in  all  procurements  Wherever  practical,  self-test  should  include  extended 
versions  suitable  for  calibration  analysis 
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TABLE  A-2.  SUMMARY  TASK  DESCRIPTION  OF  JLC  PANEL  ON  AUTOMATIC  TESTING 
(MANAGEMENT  AND  ACQUISITION  SUPPORT  AREAS) 


TITLE 

DESCRIPTION 

SCHEDULE1 

MANAGEMENT 

Document  Review 

Review  of  existing  directives,  instructions,  reg¬ 
ulations,  and  standards  and  development  of 
recommended  changes  for  a  single  DoD  document 
package  for  automatic  testing  (AT)  considera¬ 
tions  in  weapons  system  acquisition 

2Q81C 

Document  Changes 

Military  Service  review  and  implementation  of 
needed  changes  on  the  basis  of  the  document 
review,  including  coordination  via  DMSSO 

2Q82C 

Warranty  Incentives 

Evaluation  of  incentives  or  warranties  in  AT  ac¬ 
quisition  and  development  of  associated  policies/ 
procedures  for  incorporation  in  the  DoD  docu¬ 
ment  package. 

3Q80C 

Fly  Before- Buv/Fly-OfT 

Development  of  policies/procedures  for  imple 
menting  "fly-before-buy”  and  "fly-off’  concepts  in 
AT  acquisition  and  incorporation  of  recom¬ 
mended  process  in  the  Joint  AT  Acquisition 
Guide 

2Q81C 

ATE  Workload 
Management 

Development  of  computer  based  ATE  shop  man 
agement  procedures 

2Q85 

DID  List 

Compilation  and  publication  of  essential  docu 
mentation  requirements  to  be  procured  with  AT 
systems. 

3Q84 

DID  List 

Development  and  publication  of  DID  Guide  for 
AT  acquisition 

4Q84 

Corporate  Memory 

Identification  of  individual  Military  Service  orga 
nizations  responsible  for  "corporate  memory”  in 
AT  and  development  of  Military  Service  agree 
ments  for  exchange  of  information. 

3Q81C 

Corporate  Memory 

Evaluation  of  current  Military  Service  ap 
proaches  to  the  development,  maintenance,  and 
utilization  of  "corporate  memory”  and  identi 
fication  of  possible  improvements 

2CJ83 

Computer  Acquisition 
Interfaces 

Development  of  a  coordinated  ATE  computer 
acquisition  policy,  resolving  the  conflicting  regu 
lations  regarding  weapons  system/ ATE/ADPE 
computers,  and  coordination  of  recommendations 
with  the  Joint  Technical  Coordinating  Group  on 
Computer  Resources  Management. 

4Q80C 

1  Actual  ("C”)  or  scheduled  task  completion;  '*  4-  ”  if  continuing  task 
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TABLE  A-2.  SUMMARY  TASK  DESCRIPTION  OF  JLC  PANEL  ON  AUTOMATIC  TESTING 
(MANAGEMENT  AND  ACQUISITION  SUPPORT  AREAS)  (CONTINUED) 


TITLE 

DESCRIPTION 

SCHEDULE' 

AT  Coordination 

Coordination  between  the  JLC  AT  Program  and 
the  NSIA  AT  Project. 

4Q85 

Program  Benefits 

Assessment  of  benefits/effectiveness  of  the  JLC 
AT  Program 

4Q82 

ACQUISITION  SUPPORT 

Terminology 

Compilation  and  revision  of  Joint  Service  termi- 
nology  standard,  MIL-STD-1309C 

4Q83 

AT  Information 
Exchange 

Examination  of  existing  ATE  information 
systems  and  publication  of  Reference  Guide, 
including  Military  Service  Focal  Points,  DID 
Listing  (see  above),  and  Corporate  Memory  Struc 
ture  listing  (see  above) 

4Q83 

AT  Newsletter 

Publication  of  AT  Newsletter  to  improve  commu 
nication  among  the  AT  community 

Quarterly 

BIT  Design  Guide 

Development  and  publication  of  a  Joint  Service 
BIT  Design  Guide,  using  the  Navy’s  updated 
guide  as  a  baseline 

3Q82C 

Testability  Guide 
(Electronic) 

Development  and  periodic  updating  of  a  Joint 
Service  Electronic  Systems  Testability  Guide, 
using  the  MATE  Testability  Design  Guide  and 
the  RADC  Testability  Notebook  as  baseline 
documents 

2Q85 

Testability  Guide 
(Nonelectronic) 

Development  and  publication  of  a  Joint  Service 
Nonelectronic  Systems  Testability  Guide 

3Q86 

Testability  Specification 

Development  and  publication  of  a  MILSPEC 
describing  numerical  testability  parameters  and 
interfaces  required  in  electronic  system  design 

2Q84 

Testability 

Design  Review 

Development  of  a  design  review  checklist  per¬ 
mitting  a  technical  audit  of  compliance  with 
testability  requirements  and  incorporation  of 
checklist/  procedures  in  MIL-STD-1521 A 

4Q87 

BIT  Improvement 

Development  of  improved  procedures  for  speci¬ 
fication  and  evaluation  of  BIT  for  weapons 
systems.  Revision  of  MIL-STD-471B  for  demon¬ 
strating  testability  attributes  Publication  of 
military  handbook  addressing  Testability  Analy¬ 
sis. 

4Q84 

1  Actual  ("C”)  or  scheduled  task  completion,  "  +  ”  if  continuing  task 


TABLE  A-2.  SUMMARY  TASK  DESCRIPTION  OF  JLC  PANEL  ON  AUTOMATIC  TESTING 
(MANAGEMENT  AND  ACQUISITION  SUPPORT  AREAS)  (CONTINUED) 


TITLE 

DESCRIPTION 

SCHEDULE' 

MIL-STD-1388 

Coordination  of  JLC  AT  Panel  comments  on  the 
revised  LSA  standard. 

4Q82C 

LCC  Models 

Development  of  a  complete  set  of  tools  and  models 
needed  to  produce  optimum  AT  support,  using  the 
MATE  Guides  as  a  point  of  departure  and  focus¬ 
ing  on  Standard  Joint  Service  Life  Cycle  Cost 
models 

1Q86 

TPS  Design  Handbook 

Development  and  Publication  of  a  Joint  Service 
TPS  Design  Handbook 

3Q85 

ATG  Selection  Guide 

Review,  completion,  and  publication  of  the  Navy- 
prepared  Digital  ATG  Selection  Guide  as  a  Joint 
Service  guide 

4Q80C 

TPS  Acquisition  Guide 

Development,  coordination,  and  publication  of  a 
TPS  Acquisition  Guide,  using  the  MATE  Guide 
as  a  baseline 

2Q85 

TPS  V&V  Guide 

Development,  coordination,  and  publication  of  a 
TPS  Validation  and  Verification  Guide,  using  the 
MATE  Guide  as  a  baseline. 

1Q85 

MIL-STD-881 

Revision  of  the  current  work  breakdown  strue 
ture  for  defense  material  items  (MIL-STD-881)  to 
include  TPSs  and  computer  software  items 

1Q83 

ATLAS  Source  Code 

Development  and  implementation  of  procedures 
for  control  and  verification  of  ATLAS  TPS  source 
code  by  means  of  an  automatic  syntax  analyzer 
Creation  and  maintenance  of  a  library  of  ATLAS 
Syntax  Comparator  versions,  indexed  to  the 
systems/programs  using  each  version 

4Q84  + 

Hardware  Interface 

Development  of  specifications  for  standardizing 
hardware  interface  adapters  between  the  UUT 
and  ATE,  starting  with  MATE  specifications  and 
examining  feasibility  of  a  Joint  Service  specifica¬ 
tion. 

2Q83 

Sensors 

Review  of  the  state-of-the-art  of  sensors,  updating 
of  the  Sensor  Handbook,  and  identification  of 
R&D  needs  for  new  and  improved  sensors 

4Q84  -h 

Training  Courses 

Development  and  presentation  of  AT  related 
training  courses 

4Q85  -f- 

1  Actual  ("C”)  or  scheduled  task  completion;  "  +  ”  if  continuing  task 


TABLE  A-2.  SUMMARY  TASK  DESCRIPTION  OF  JLC  PANEL  ON  AUTOMATIC  TESTING 
(MANAGEMENT  AND  ACQUISITION  SUPPORT  AREAS)  (CONTINUED) 


TITLE 

DESCRIPTION 

SCHEDULE1 

TRD/TPS  Standard 

Analysis  of  existing  documentation  require¬ 
ments,  MIL-STD-1519  (USAF),  MIL-STD-1345B 
(Navy),  MIL-STD-2076  (AS),  MIL-STD-2077 
(AS),  for  the  purpose  of  combining  all  into  a 
single,  Joint  Service  standard. 

1Q84 

ATE  Selection 
Methodology 

Development  of  ATE  selection  methodology  as 
part  of  the  MATE  program;  Military  Service 
review  of  methodology;  and  publication  as  a  Joint 
Service  guide. 

4Q85 

Commercial  Test 
Equipment 

Survey  of  acquisition  and  life  cycle  support  of 
ATE  built  with  commercial  test  equipment 
(CTE);  development  of  recommended  changes  in 
policies/procedures  that  will  rectify  any  problems 
identified;  and  preparation  of  a  program  of  action 
and  milestones  to  implement  recommended 
changes. 

3Q83 

TABLE  A-3.  SUMMARY  OF  HIGH-PRIORITY  RESEARCH  AND  DEVELOPMENT 
RECOMMENDATIONS  BY  THE  INDUSTRY  AD  HOC  ATE  PROJECT 

FOR  THE  NAVY 


TOPIC 

ISSUE 

RECOMMENDATION 

Software 

ATE  software  is  a  major  contributor  to  the 
lifecycle  cost  of  prime  weapons  systems. 

Identify  the  c  <  ;t  drivers  and  control  their 
effect  on  TPS  development  and  mainte 
nance.  Improve  TPS  quality  by  improving 
the  acquisition  process  for  test  requirements 
data 

ATE  software  management  is  inadequate, 
with  particular  problems  in  configuration 
management,  quality  control,  and  feedback 
of  TPS  deficiencies. 

Improve  ATE  software  management  and 
control  methods.  Improve  feedback  meth¬ 
ods.  Standardize  ATE  software  tools,  tech¬ 
niques,  and  interfaces.  Improve  operational 
aspects  of  ATE  software  to  reduce  run  time 
of  operating  software  as  well  as  TPSs. 

Automatic  Test 
Generators  ( ATGs  I 

The  trend  toward  LSI  is  out  distancing  the 
capabilities  of  digital  ATGs  to  generate 
quality  test  programs. 

Because  of  wide  variety  of  analog  equipment 
that  will  never  be  replaced  by  digital  elec 
tronics,  there  is  a  need  for  analog  ATGs. 

Initiate  R&D  program  to  advance  the  state 
of  the  art  in  digital  ATGs 

Sponsor  R&D  program  for  analog  ATGs. 

Design  for 
Testability 

Traditional  design  methods  and  acquisition 
procedures  do  not  result  in  supportable  sys¬ 
tems  at  affordable  costs.  A  new  methodology 
is  needed  to  achieve  testability. 

Sponsor  a  continuous  R&D  program  to 
develop  new  BIT  techniques  and  the  means 
for  implementing  these  at  the  system  level. 
Develop  testability  specifications,  along 
with  the  means  for  quantitatively  measur¬ 
ing  this  parameter.  Develop  LCC  model 
methodology  as  a  tool  for  assessing  cost- 
effectiveness  of  testability.  Improve  acquisi¬ 
tion  management  by  motivating  contractors, 
enforcing  LSA,  and  instituting  testability 
audits/reviews. 

Nonelectronic 

Equipment 

Monitoring 

Shipboard  machinery  maintenance  is  a 
costly  burden,  is  time-consuming,  adversely 
affects  crew  morale,  is  unreliable,  and  fre 
quently  causes  damage. 

Develop  prototype  monitoring  and  diagnos¬ 
tic  systems  for  evaluation  at  sea  Evaluate 
adequacy  of  existing  commercial  sensors, 
modify  Navy  specifications,  establish  QPLs 
for  families  of  machinery  monitoring  -*en 
sors,  and  identify  R&D  needs  Develop 
guidelines  and  standards  to  ensure  compa¬ 
tibility  between  monitoring  systems  and 
Ships  Data  Multiplexing  System.  Expedite 
test  and  e\  aluation  of  the  l.M  2500  and  FT  9 
engine  condition  monitoring  systems 
Develop  and  evaluate  prototype  diesel 
engine  condition  monitoring  system  for  the 

LST  1179  ('lass. 

New 

Technology 

In  the  past,  weaponssvstem  acquisition deci 
sions  have  been  based  on  potential  perfor 
mance  effectiveness  and  initial  costs, 
ignoring  the  fact  that  life  cycle  costs  are 
dominated  by  operational  and  maintenance 
costs.  Introduction  of  new  technology  can 
worsen  life  cycle  costs,  if  risks  have  not  been 
adequately  assessed 

Develop  objective  methodologies  to  assess 
new  technologies  and  guide  their  implemen¬ 
tation,  including  identification  of  risks,  R&D 
needs,  available  alternatives.  ILS  require 
ments,  and  state  of  associated  manufactur¬ 
ing  technology.  Establish  test  standards  and 
methodologies  for  microprocessors  and  mass 
memory  devices  Develop  advanced  BIT 
capabilities  for  lasers,  fiber  optics,  and  solid 
state  imaging  devices  to  preclude  complex 
alignment  problems  Initiate  R&D  for  the 
application  of  microprocessors  to  enhancing 
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TABLE  A  3  SUMMARY  OF  HIGH-PRIORITY  RESEARCH  AND  DEVELOPMENT 
RECOMMEN  DATIONS  BY  THE  INDUSTRY  AD  HOC  PROJECT 
FOR  THE  NAVY  (CONTINUED) 


RECOMMENDATION 


New 

Technology 

(Continued! 


Education. 
Training,  and 
Management 


Advanced 

ATE 

Concepts 


There  is  a  seru  us  void  in  the  training  of 
decisionmakers  within  Navy  and  industry 
Navy  acquisition  managers  have  a  limited 
understanding  of  the  technical  capabilities 
and  limitations  of  automatic  testing  Navy 
ILS  planners  have  a  limited  understanding 
of  the  relationships  between  ATE  and  main 
tenance  concepts.  Contractor  ATE  develop¬ 
ers  have  a  limited  understanding  of  the  skill 
levels  of  Navy  technicians  operating/main¬ 
taining  ATE  and  the  way  ATE  is  utilized  in 
the  Navy 


The  Navy  has  yet  to  optimize  its  exploitation 
of  ATE  technology  and  can  learn  valuable 
lessons  from  industry.  The  Navy's  dilemma 
between  low  throughput  of  the  expensive 
general-purpose  supertester  versus  prolifer 
ation  of  peculiar  test  equipment  usurping 
available  space  must  be  resolved  through 
adopting  a  new  approach. 


BIT  in  future  systems.  Sponsor  R&D  pro¬ 
gram  on  automated  test  techniques  related 
to  RE  and  microwave  technology. 


Develop  courses  suitable  for  indoctrinating 
Navy  and  contractor  personnel  in  ATE 
design,  procurement,  and  use  Develop  and 
implement  a  training  program  for  the  wea¬ 
pons  system  acquisition  community  inaval 
officers  and  civilians!,  matching  personnel 
classifications  to  the  spectrum  of  courses 
developed.  Develop  selection  criteria  for 
new  instructional  media  and  training  aids, 
and  apply  new  training  technology  in  train¬ 
ing  ATE  opera tors/maintainers. 


Sponsor  R&D  program  to  define  and  develop 
the  next  generation  or  family  of  ATE 
Sponsor  studies  to  examine  the  optimum  use 
of  ATLAS  and  OPAL  test  languages  and  the 
potential  of  graphic  programming  Develop 
and  standardize  eight  families  of  ATE  build 
ing  blocks  (power  supplies,  switching  mod 
ules,  microwave  signal  synthesizers,  scan 
converters,  mass  memory  testers/logic  ana 
lyzers,  high-power  test  stations,  ruggedized 
key  commercial  instruments,  and  operator 
control/display  stations!  Sponsor  a  long 
range  continuing  R&D  program  to  develop 
advanced  test  techniques  for  BIT  and  ATE 
application.  Sponsor  a  5-vear  R&D  program 
to  examine  and  improve  the  ATE  man- 
machine  interaction  Establish  an  ATE 
technology  center  to  assimilate  the  wealth  of 
technical  data,  coordinate  R&D,  and  assess 
impacts 


2l 


TABLE  A-4  SUMMARY  OF  AIR  FORCE  TMDE  MANAGEMENT  INITIATIVES 


ISSUE 

RECOMMENDED  ACTION 

"Lack  of  SE  planning  results  in  proliferation, 
increased  costs,  and  deployment  constraints." 

Develop  SE  Master  Plan,  including  action  plan  to  achieve  stan 
dardization  for  each  SE  category,  updated  annually.  Revise 
AFR  800- 12  to  standardize  SE  definitions  and  to  require  all  major 
commands  support  the  Master  Plan.  Establish  separate  program 
element  for  common  SE  to  fund  common  SE  development. 

"Organizational  structure  for  managing  SE  is 
fragmented,  with  no  clear  lines  of  respon 
sibility  " 

Create  SE  broker/advocate  function,  responsible  for  SE  planning. 
MIL-HDBK-300  support,  and  improving  SE  procedures;  assign 
function  to  Air  Force  Acquisition  Logistics  Center  fAFALC) 
Revise  AFSCR/AFLCR  800-5 

"Planning  of  SE  acquisition  strategies  is  not 
always  effective." 

Revise  policy  to  improve  scrutiny  of  SE  in  the  acquisition  process 
i  AFR  800-2,  AFR  800  5.  AFSCR  70-2,  AFSCR  70-7,  AFLCR65-5. 
AFSC/AFLCP 800  34).  Increase  use  of  FAR  15  213.  with  delega 
tion  of  authority  below  Secretarial  level,  and  multiyear  contracts 
for  TMDE.  Adopt  AFMAG1  recommendations  pertaining  to  tech¬ 
nical  data  acquisition 

"SE  is  not  adequately  emphasized  in  program 
management  documents,  especially  Program 
Management  Directives  i  PMDs) " 

Revise  USAFHOI  800-2  and  AFR  800-2. 

"MIL  HDBK  300  does  not  adequately  support 
L’SAF  requirements  for  collecting  and  distrib¬ 
uting  SE  information  required  by  Program 
Managers  and  Air  Force  contractors  " 

Transfer  management  responsibility  for  MIL  HDBK  300  from 
CASO-  to  SE  broker/advocate.  AFALC  Implement  JLC  Panel  on 
AGSE  recommendations  to  improve  MIL-HDBK  300  and  develop 
automated  screening  process. 

"SE  is  frequently  late  for  unit  conversions  and 
unit  or  depot  activations  ” 

Revise  budgeting  procedures  (AFLCR57  2,  AFLCR  57  1  1 1.  use 
interim  release  procedures  i AFLC/AFSC'P 800-341,  retain  SE  of 
deactivating  units  ( AFLCR  57  2).  plan  depot  maintenance  earlier 
( AFSC/AFLCR  800  32),  identify  shortfalls  in  decision  tree  analy 
sis  processtAFLCR66-75).  and  satisfy  early  data  requirements  by 
I.SAR  data  sheets  I  AFSC/AFLCR  800  30 1 

"Guidance  and  planning  for  calibration  is  inade¬ 
quate  " 

Revise  AFR 800-12  and  AFSC/AFLCP  800-34.  Direct  AGMC  to 
promulgate  guidance  on  calibration  concepts  Revise  directives  to 
ensure  TMDE  is  supportable  (AFR  400  37.  AFR  74  2. 
AFSCR 67  8,  AFSCR67  6i 

"Failure  to  integrate  SE  design  and  develop 
ment  into  the  systems  engineering  process 
contributes  to  SE  proliferation  and  increases 
long-term  support  costs." 

Revise  AFR  800  12  and  AFSC/AFLCR  800  5  to  require  SE 
considerations  be  part  of  early  system-level  trade  studies,  with 
emphasis  on  reducing  or  eliminating  SE  Revise  MIL  STD  1521  A, 
AFLC/ AFSCR  800-24.  and  AFLC/AFSCP  80(1  34 

"Current  LSA  and  SERD  guidelines  and  proce¬ 
dures  are  not  effective  in  identifying  and 
reporting  SE  requirements." 

Immediately  develop  a  modern  AIM*  capability  to  proo-ss  LSA 
data,  interfaced  with  an  automated  MIL  HDBK  300  and  m 
automated  engineering  drawing  retrieval  system  Rev  n* 

AFR  800-8  la  legitimize  use  of  LSAK  "E“  sheet  as  an  .liter  naf  i\  e  to 
SERD,  eliminating  redundant  data  bases  and  reporting  require¬ 
ments.  but  modify  "E”  sheet  to  include  narrative  text  and 
references  to  engineering  draw  mgs/Iists  defining  proposed  SE 
Require  LSA  programs  to  include  SE  tor  SE  other  than  tools  and 
plant  equipment.  Develop  a  common  SERI)  Hacking  system  for 
use  by  all  SE  requisition  agents  Simplify  SERD  rev  lew  process 

"Improper  SE  design  and  specification  practices 
impact  performance,  increase  costs,  and  reduce 
competition." 

Emphasize  all  programs  on  the  design,  specification,  and  configu¬ 
ration  management  nt  SE.  including  DT&E,  OT&E.  and  technical 
order  y  alidation  and  verification.  Develop  standardized  CDKL  for 
.icquisition  of  engineering  data  to  be  used  in  re  procure  merit 

The  AFMAC. study  of  .spare  parts  problems.  conducted  in  1983,  attributed  many  problems  to  SE  acquisition  problems 
-C ASO  is  an  AFLC  activity  located  in  Battle  Creek.  Michigan.  The  Logistics  Management  Institute's  review  of 
programming  data  indicates  that  C ASO  has  been  spending  less  than  one-half  man-vear  annually  on  keeping  MIL  HDBK  300 
updated. 


TABLE  A-4  SUMMARY  OF  AIR  FORCE  TMDE  MANAGEMENT  INITIATIVES  (CONTINUED 


ISSUE 

RECOMMENDED  ACTION 

"Support  for  SE  is  expensive  and  frequently 
late.” 

Revise  AFR800-12  and  AFSC/AFLCR  800  5  to  require  support 
considerations  for  SE,  including  application  of  ILS  (AFR8OO-81 
and  LSA  (MIL-STD- 1388)  to  SE  acquisition. 

"Funding  for  common  SE  is  frequently  late  and 
inadequate,  and  lacks  visibility.” 

Continue  emphasis  on  program  "baselining”  1  Acquisition  Im 
provement  Initiative  #30).  Establish  a  new  program  element  for 
development  of  common  SE.  Program  funds  for  common  SE 
through  the  POM  cycle  based  on  forecasted  requirements  just  as 
peculiar  SE  funds  are  programmed.  Implement  regulatory 
changes  requiring  weapons  system  SPOs  to  program  common  SE 
funds  for  initial  acquisition,  rather  than  AFLC  and  the  ALCs 
(AFR  800-12.  AFM  172  1). 

"Technical  Orders  for  SE  are  frequently  late  and 
inadequate.” 

Enforce  AFR  8-2  and  TO  00-51,  requiring  delivery  of  the  TOs  for 

SE  at  the  same  time  the  equipment  is  delivered. 

"SE  acquisition  methodology  needs  to  be  im 
proved  to  properly  incentivize  contractors." 

Require  each  program  manager  to  plan  for  SE  incentives 
Establish  requirement  in  PMD  and  acquisition  plan 

"SE  is  frequently  acquired  by  personnel  who 
have  not  received  appropriate  training." 

Direct  AF1T  to  expand  coverage  of  SE  in  its  program  management 
and  logistics  courses  Develop  an  SE  Acquisition  Handbook  to 
replace  the  outdated  Aeronautical  Svstems  Division  Pam 
phlet800-I9.  Ensure  that  personnel  involved  in  SE  acquisition 
receive  requisite  training 

"The  estimated  price  for  SE  often  greatly 
exceeds  the  intrinsic  value  of  the  item  itself." 

(Report  lists  17  recommendations  that  are  beyond  scope  of  our 
study.) 

"Lack  of  visibility  of  small  dollar,  sole  source 
procurement  prices  increases  probability  of  over 
pricing  on  SE.” 

(Beyond  scope  of  our  study.) 

"There  is  no  standard  management  information 
and  control  system  to  aid  SE  managers  in  the 
acquisition  of  SE.” 

Develop  an  interactive,  real  time  data  system  for  use  by  Air  Force 
and  contractors  to  pass  and  track  SE  data. 

"The  information  systems  used  to  acquire  and 
control  common  SE  are  outdated  and  inaccurate, 
degrading  the  Air  Force's  ability  to  effectively 
manage  these  assets." 

Continue  current  efforts  to  improve  C013  (Support  Equipment 
Acquisition  and  Control  System)  and  the  interfaces  among  the 
hodgepodge  of  computer  systems  constituting  the  AFEMS 
Initiate  actions  to  develop,  lest,  and  implement  an  on-line  real 
time  replacement  for  AFEMS  Revise  the  AFLC  AFEMS  manage 
ment  structure  to  bring  all  AFEMS  programmers  and  managers 
under  control  of  j  single  manager 

GLOSSARY  OF  TERMS  USED  IN  APPENDIX 


ADP 

ADPE 

AFALC 

AFEMS 

AFIT 

AFLC/AFSCP 

AFLCR 

AFMAG 

AFR 

AFSCR 

AGMC 

AGSE 

ALC 

AT 

ATE 

ATG 

ATLAS 

ATPG 

BIT 

CASO 

CDRL 

COTS 

CSS 

CTE 

DID 

DMSSO 

DoD 

DT&E 

EDP 

FAR 

HOI 

1C 

ID 

IEEE 

IFTE 

ILS 

J  LC 

LCC 

LRU 

LSA 

LSAR 

LSI 

MATE 

METCAL 

MIL  HDBK 

MILSPEC 

MIL  STD 


automatic  data  processing 
automatic  data  processing  equipment 
Air  Force  Acquisition  Logistics  Center 
Air  Force  Equipment  Management  System 
Air  Force  Institute  of  Technology 

Air  Force  Logistics  Command/Air  Force  Systems  Command  Pamphlet 

Air  Force  Logistics  Command  Regulation 

Air  Force  Management  Analysis  Group 

Air  Force  Regulation 

Air  Force  Systems  Command  Regulation 

Aerospace  Guidance  and  Metrology  Center 

aviation  ground  support  equipment 

Air  Logistic  Center 

automatic  testing 

automatic  test  equipment 

automatic  test  generator 

Abbreviated  Test  Language  for  All  Systems 

automatic  test  program  generator 

built-in  test 

Cataloging  and  Standardization  Office 
contract  data  requirements  list 
commercial  off-the-shelf 
Consolidated  Support  System 
commercial  test  equipment 
Data  Item  Description 

Defense  Materiel  Specifications  and  Standards  Office 

Department  of  Defense 

Development  Test  and  Evaluation 

electronic  data  processing 

Federal  Acquisition  Regulation 

headquarters  operation  instruction 

integrated  circuit 

interface  device 

Institute  of  Electrical  and  Electronics  Engineers 

Intermediate  Forward  Test  Equipment 

integrated  logistic  support 

Joint  Logistics  Commanders 

life  cycle  cost 

line  replaceable  unit 

logistics  support  analysis 

Logistics  Support  Analysis  Record 

large  scale  integration 

Modular  Automatic  Test  Equipment 

metrology  and  calibration 

Military  Handbook 

Military  Specification 

Military  Standard 


National  Bureau  of  Standards 

not  operationally  ready  due  to  maintenance 

not  operationally  ready  due  to  supply 

National  Security  Industrial  Association 

Operational  Procedures  Analysis  Language 

Operational  Test  and  Evaluation 

Program  Management  Directive 

Program  Objective  Memorandum 

quality  assurance 

qualified  products  list 

Rome  Air  Development  Center 

research  and  development 

radio  frequency 

support  equipment 

support  equipment  recommendation  data 
system  program  office 

test,  measurement,  and  diagnostic  equipment 

Technical  Order 

test  program  set 

test  requirements  analysis 

test  requirements  document 

U  S  Air  Force 

unit  under  test 

very  large-scale  integration 

validation  and  verification 

weapon  replaceable  assembly 
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lupport.  Volume  III  describes  how  the  Military  Departments  are  organized  to  carrv  out  that  management  and  support;  and  Volume  IV  reviews  and 
issesses  the  adequacy  of  DoD  policy 
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